Document Processing with Artificial Intelligence

A perfect, automated process is often interrupted by the need for a human to intervene. The person becomes a vital link in the chain between separate computer systems. The overall process becomes hard to scale, expensive and unreliable.

When the number of documents grows, finding information becomes impractical. Cross-referencing multiple files or combining information from a large set of documents is unworkable.

Most organisations have simply too many documents for people to make sense of.

When documents are spread across different systems, different folders, and different locations, just finding the right document takes too much time. And who doesn’t love SharePoint, right?

AI can be our modern labour-saving device in the office. Back in the 1960s, we were promised more leisure time through technology. Perhaps AI will save us from the drudgery of searching for and reading long documents.

Machine learning models are the magic missing links in a document processing pipeline.

Reading the documents

Modern document intelligence solutions are a natural progression of optical character recognition (OCR) that became popular in the 1970s. The power of modern computer hardware and software allows far more sophisticated natural language processing. Even documents written by hand in a range of different languages can be understood.

A training process is used to create a machine-learning model. Many pre-trained document models are available for standard documents such as passports and identity cards. Less standard documents that follow a typical format, such as invoices and expense receipts have ready-to-use models available.

Models can be created for less common documents that are specific to particular use cases. These models are typically trained with supervised learning. A human will provide a range of sample documents and guide the model towards the correct answer. The model is then refined with positive and negative feedback based on its future answers.

After a document has been read (or parsed), information is extracted as plain text, key/value pairs and tables of data. It can then be stored for later processing.

Understanding the information

Converting documents into structured or semi-structured storage is an important first step. Unlocking the value of this information is where artificial intelligence starts to shine.

Search indexes are created from data in the documents. This works in much the same way as your favourite search engine will index the web.

Your documents can be processed automatically. This is sometimes referred to as document cracking or enrichment. For example, extracting key phrases across documents, generation of automatic document summaries or creating automatic FAQs.

Your end users can explore (or search) the documents using natural conversation and easy-to-use interfaces. Subject to the appropriate security and access restrictions.

Intelligent document processing is usually thought of as a pipeline through which the information flows. Each step along the way has a particular responsibility.

• Step 1 - Collect your documents

  Finding important company information is often like a really disappointing treasure hunt. Documents are often buried within network shared and folders specific to teams or projects. There is usually any number of SharePoint sites to navigate. Sometimes, important files simply live in email inboxes.

  Automated document processing systems can be configured to connect directly to different file sources. This allows for a largely automated discovery process.

  Documents can be collated into a common storage area (similar to the concept of a data lake). When a new document is stored, an event triggers the data extraction process. This approach can work well from a security perspective. It also simplified managing the document lifecycle for keeping the documents for a defined period after processing, or removal if required. The document lake is also useful for ongoing training of the models and investigating any issues.

• Step 2 - Classify and extract the information

  Recent innovations in machine learning and AI have transformed document recognition and understanding.

  Azure Cognitive Services allow documents to be classified according to their content. Document layout can be recognised and the text parsed. Form Recognizer automatically understands printed text and handwriting – in a range of languages. Structured data can be extracted from lists and tables.

  The text itself can be interpreted to understand its semantics. The sentiment of text can be converted to values that can be readily aggregated and compared. Summaries and abstracts can be generated.

  AWS Textract can extract text and handwriting content within documents. AWS Comprehend is able to process the text, extract key phrases and identify important entities mentioned. PII is automatically identified and can be redacted. A variety of source document types are supported, including PDF, Word documents and images.

  Choosing Azure and AWS services will depend on a variety of factors. Some organisations find the nature of their documents will work better with some models than others. In many cases, the decision is driven by any existing cloud infrastructure.

  Existing line-of-business applications can be enhanced with intelligent document systems by connecting them to Azure, AWS and other services. These AI services provide SDKs and APIs that allow developers to interact with them from new and existing code.

• Step 3 - Store the extracted information

  The key to creating a useful document intelligence process is storing the information so it can be easily read. Various storage options are available to make searching easy and quick.

  Structured data (that is, where the format is well-defined) can be stored in ordinary relational databases such as Azure SQL Database or Amazon Aurora. Unstructured and semi-structured data (where the format isn’t known or may change over time) can also be stored effectively. Azure Cosmos DB, Amazon DocumentDB and Amazon DynamoDB are excellent choices.

  Search indexes are also an effective way to store large amounts of unstructured information. For example, Azure Cognitive Search and Amazon CloudSearch are designed for storing and searching across large amounts of complex information.

• Step 4 - Verification and review

  The accuracy of AI solutions is improved by continuous feedback. Pipelines should be instrumented to provide performance telemetry. Metrics might include indicators for levels of precision in the matches, how many predictions have been recalled etc. Alerts are useful to help resolve issues quickly.

  Keeping a human in the loop to review AI predictions is helpful. For critical decisions or high-value data, a human review of all predictions should be considered. All machine learning systems can benefit from ongoing feedback. Correcting errors and confirming predictions will enhance the learning models and improve the confidence scores.

Automate your document pipelines

Documents often become a manual step that breaks an otherwise automated process. A human might have to save files from an email or download them from a website. Sometimes the files simply need to be moved. Sometimes they might need to be manually verified. Sometimes they might need to be corrected or enhanced.

A "swivel-chair" style manual step can be expensive, unreliable, and difficult to scale. If the person isn’t available, the process can stall without warning.

Document intelligence can remove the bottlenecks and allow processes to scale efficiently. Some document automation pipelines might just require some straightforward steps. More advanced processing is sometimes required to parse the files, understand the content and extract key sentiments.

Feed your knowledge base

Useful content is often buried away in documents that people don’t even know exist, let alone have time to read.

Intelligent document pipelines can automate the process of summarising information and extracting the most important contents. This information can be used to generate question and answer pages, company wikis and conversational chatbots.

Automation helps to ensure that the information is extracted efficiently and reliably.

Reading handwritten documents

Not all documents are neatly arranged and typed. Documents from customers, remote workers, factory operations or client-facing people are often handwritten. While they might use a common form, the handwriting can be time-consuming to decipher.

Modern AI can read handwriting of varying quality and in a number of different languages. The text can be converted to a common language and then parsed using further AI techniques.

Handwriting recognition can improve reliability and make scaling up more cost-effective.

Help users find your documents

AI can index your documents so they can be found using natural language.

Indexing takes the information contained in your files and stores a representation of it in a database. Text can be translated and parsed for meaning (i.e. understood). Text in embedded images or even speech from videos can be extracted and indexed.

Users can search for information in the same way they might search the web using Google search. Search can include sophisticated features that users have come to expect. Autocompletion of queries, synonym matching, relevance weighting and other tunings are possible.

Human Resources have to deal with a large number of documents. Intelligent document processing can help with all stages of the employee lifecycle. The employee onboarding process depends on processing different types of documents to validate identity, right to work and qualifications. The manual process is time-consuming and often unreliable. Automation can help improve reliability, reduce the burden on HR teams and help resolve any delays.

Customer Service teams can automate the documents they receive from customers. For example, claims handling teams can work with documents from policyholders or suppliers. Automation can be useful to improve the customer experience during a claims process. Information extracted from documents can feed into a fraud detection process. Automation can reduce the burden of claims processing and reduce the time from first notification to resolution and closure.

Finance teams can automate invoices that their accounts payable department receives to improve accuracy and reduce manual effort. Accounts receivable departments can automate the processing of customer documents such as remittance advice and purchase orders. Other customer documents such as proof of identity and registration documents can also be processed.