Intelligent data capture: rubbish in, rubbish out

The latest intelligent automation technologies can do amazing things for your business - streamlining and connecting everything from invoice processing to complaints management to KYC. But there's a key dependency. Intelligent automation solutions are only as good as the data you capture and feed into them. We explain how you can avoid sabotaging automation projects from the outset by deploying information capture and recognition technology in a truly intelligent way.

In computer science, rubbish in, rubbish out is a founding principle. It means that the quality of output from any operation is governed by the quality of the input. The concept is just as important in process automation. But it's all too often overlooked in the excitement of exploring all the futuristic potential of ground-breaking technologies such as artificial intelligence (AI), machine learning (ML) and robotic process automation (RPA).

The key word here is potential. You can adopt the most capable and sophisticated intelligent automation solution but if the data you feed in is inconsistent, incomplete, corrupted or inaccurate, you won't get usable business information out of it. At best, you'll need to make a lot of manual interventions. At worst, you'll bake in errors and magnify them down the line. Either way, the benefits of the tech will elude you - they're no more than potential without good data.

You need to sort your rubbish first

Intelligent automation technologies offer organisations in all sectors the chance to boost productivity, streamline and cut costs. But experience shows us that not everyone is getting the value from it that they expected.

Successfully deployed, automation means your business is getting great efficiency, productivity and value, without hassle. You're getting intelligent and trusted information out that speeds up and streamlines your operation.

This is no pipe dream. It's realistic and very possible. Amongst our clients, a transport group reduced their invoice processing time by 50%, freeing up staff to focus on more valuable work. A utility provider completely automated the processing of 2,500 timesheets per week with a significant reduction in errors. A global finance organisation automated the matching of 250,000 invoices in a variety of languages across 55 operating units. And a global retail group reduced their use of paper in the office by 43% by automatically extracting key data from electronic documents at source.

But not everyone experiences this kind of success.

It's all too common for organisations to approach us to investigate why an existing solution throws out so many exceptions that staff needs to validate or correct. Or we're called on to review or replace an application that hasn't delivered the insights or faster responses that were promised in the business case.

Automation performance depends on the quality and type of data input

Why doesn't every organisation experience a transformative benefit from their automation? Why isn't everyone doing it immediately, when these kinds of gains and savings are possible? In our experience, many of the barriers and challenges that clients shy away from are to do with the difficulty of getting their data into the system in a sufficiently complete and accurate way.

Although intelligent data capture is a significant step forward, the technology is not foolproof.

For the biggest automation gains, you need to be able to capture and input data that isn't completely predictable. In the early days of optical character recognition, technology was capable of extracting data from structured sources, where data is in a particular format and location in a document, for example, a form with the boxes correctly filled out.

Later, smarter solutions tackled semi-structured data, where a form or document contains expected pieces of information that can be identified even if they're not in a consistent location. For example, supplier invoices, where the date, purchase order and item costs will be laid out in house style.

Intelligent data capture can handle unstructured data from a range of sources

Today, we need to work with unstructured data, where information is received in a variety of formats. Intelligent data capture is the latest evolution, leveraging the tools mentioned above and combining them with artificial intelligence and pattern recognition technologies to drive higher accuracy through each step of the process.

For example, in a customer complaints process, customers may email or write in and provide a range of comments and details about different issues that need to be handled by particular teams. They may supply receipts or documents in support of claims. You need intelligent data capture technology that can handle this sophistication.

AI and machine learning (ML) have increased the capabilities of intelligent data capture solutions. Through pattern recognition, we can understand the document type and know what to do with the information contained. And the latest tools continually learn over time requiring less feedback from staff to hone its ability to identify data in different locations and expressions. This is particularly useful for unstructured data sources.

Crucially, the efficacy of these solutions depends on the way they're deployed. Properly optimised, they're highly accurate and reliable, correctly filtering the data they acquire. 

Using intelligent data capture to solve critical business issues

The potential use cases of intelligent data capture are endless. Here are five smart ways our customers have recently applied the approach:

• Fully automated, digital mailroom reads unsolicited customer letters and routes them to the right department for response and action
• Intelligent system matches invoices to purchase orders across different languages and notation systems (including Asian and East European script)
• Smart solution decodes poor handwriting into a machine-readable format for further processing via workflows
• Automated reader identifies sensitive financial information in documents and produces a redacted, cleansed version to meet GDPR requirements
• Intelligent capture platform extracts information from a range of media including SMS (text), social media messages, scanned documents, emails and attachments.

Making it happen: practical tips for optimising and embedding effective data capture 

Before beginning your transformation journey, it is important to make an informed evaluation of your requirements versus available capture technologies. This can be a daunting prospect for those not adept in this area.

From our decades of experience, we've learned a lot about the right way to approach and deploy data capture solutions. Use our checklist of key issues to identify your needs, select the most suitable data capture tools and maximise your return on investment:

• Level of accuracy
  No software will deliver 100% accuracy all the time, especially where input data is unstructured or handwritten. However, when the right tools are properly optimised, they're highly accurate and reliable. They flag only genuine exceptions for manual intervention, and loop in the learning to make sure they recognise similar data next time. This avoids the AI spectre of incorrect data being accidentally legitimised, amplified and built on to create even greater inaccuracies.
  
• Integration
  Data capture can be deployed as a standalone tool, but more commonly it is the starting point for a broader intelligent automation solution. Time spent upfront understanding integration challenges with other IT systems and databases will help to avoid painful integration issues during and after deployment.
  
• Structured vs unstructured data
  From the outset, you need to be clear whether data is to be captured from uniform document types in a standard format or if there is significant variation. The quality and design of documents to be processed should be assessed upfront and optimised where this is an option. By tweaking document layouts even higher recognition rates can be delivered. A more powerful and sophisticated capture engine will be required to handle highly unstructured data sources.
• Security
  Data extracted using intelligent data capture tools often contains highly sensitive personal or financial data. You need to understand where this data will be stored, and whether there are any potential security implications. This may mean you need to deploy automated redaction tools.
• Validation
  Business validation rules are a critical component of any data capture solution. After all, what good is the data you are collecting if you can not verify that it is correct? Validation rules can compare data extracted to an existing database or confirm that individual fields are consistent with expectations (e.g. certain numeric characters, date ranges etc). These validation rules can reduce manual user intervention and improve the accuracy of your data as it flows into other business processes.

Finding a data capture solution that precisely meets your unique needs

There's no question that intelligent automation tools are essential for present and future business success - through automating processes, cutting out manual intervention, improving accuracy and speeding up responsiveness.

Capturing data in an intelligent way is a key ingredient. If you get this right first, the world's your oyster for processing, combining and sharing the data in any way your business needs - whether it's for business strategy, finance, marketing, risk assessment, customer service or HR.

Every business is different. There's no one-size-fits-all solution, but there are smart ways to bring data together and make sure it's the right quality, taking into account the reality of the systems and data you have today and your organisation's priorities and needs.

It sounds a tall order, but it doesn't have to be. We have decades of experience applying pragmatic data capture solutions to businesses in all kinds of sectors, whether they're starting with one small area for automation or going in for a major project.

At Inpute, our expert, no-drama approach is all about doing what your business needs to get everything working smoothly. Get in touch if you'd like to hear more about optimising intelligent data capture or to get the most from your automation journey.