A document AI demo can turn an invoice into a neat table in seconds. That is a useful start, but a neat table does not tell you whether the output is ready for your accounting software, CRM or operations dashboard.
Before connecting extraction to a live system, answer a more useful question: does it capture the information your process needs, on the documents your business actually receives?
Here is a practical way to evaluate it without turning the exercise into a machine-learning project. The examples below are hypothetical; this is an evaluation guide, not a report of tests we have run.
1. Define the record you need
Start with the destination. List the fields the next system expects and what each means. For an invoice, that might include supplier, invoice number, invoice date, currency, subtotal, tax, total and line items.
Be precise about look-alike values. An invoice date is not a due date. A supplier’s address is not necessarily the delivery address. A total can be expressed in one currency while a note mentions another.
Decide how the system should represent information that is absent or unclear. An empty field with a reason is more useful than a plausible guess that nobody notices.
2. Build a small, varied test collection
Use documents you are authorized to process. Include the range your team handles, rather than selecting only clean examples that make the demo look good.
Useful variations include a digitally generated PDF, a scan, a multi-page invoice, several line items, a different supplier layout and a credit note if that is part of the workflow. Include a document with missing information too.
Keep the test collection separate from examples used to tune the extractor. Otherwise, improvements can reflect familiarity with those files rather than better handling of new documents.
For each test document, have a person record the expected values. Keep a link back to the source page so disagreements can be checked. There is no universal sample size that proves readiness; expand the collection as you discover new layouts and failure cases.
3. Score important fields separately
Avoid relying on one headline percentage. Correctly extracting many easy fields can make the overall result look strong while a critical field remains unreliable.
Google’s Document AI evaluation documentation describes comparing predicted entities with annotated test documents and reviewing metrics for individual labels as well as all labels. It uses precision and recall rather than a single accuracy score.
In plain language, ask two questions for each field: how often are the returned values correct, and how often does the system find the values it should find? Those are different questions. Returning very little can reduce wrong output while leaving your team with more missing information to resolve.
Record wrong values, missing values and extra values separately. Also note the business consequence. A supplier name that needs a small formatting correction and a wrong invoice total should not disappear into the same undifferentiated error count.
4. Check rows, pages and meaning
A document can have correct header fields and incomplete line items. Check whether every relevant page was processed and whether rows are missing, duplicated or joined incorrectly.
For a hypothetical three-page invoice, a correct final total does not prove that the items on page two were captured. Compare the extracted row list with the source, not just the summary at the top.
Agree on formatting rules before scoring. Should dates use a consistent format? Should amounts retain decimal precision? How should a credit be represented? Normalization can be useful, but it should preserve meaning rather than silently change it.
Basic consistency checks can flag a record for investigation. For example, compare the extracted amounts using the invoice’s stated rules for tax, discounts and rounding. A mismatch is a reason to inspect the document; it is not automatic proof of fraud or an excuse to invent a balancing value.
5. Measure the work left for people
Track how long a reviewer needs to correct a record and which errors keep recurring. A tool that creates an attractive table but requires extensive checking may not improve the overall process.
Make the review easy: show the extracted field beside the relevant source page and preserve the original document. Let reviewers distinguish an unreadable value from a field that is genuinely absent.
If a tool exposes confidence scores, treat them as something to evaluate on your documents. A high score is not a guarantee that a value is correct, and a threshold should not be chosen just because it produces fewer warnings.
6. Connect gradually
Begin by exporting proposed records for comparison. Next, test the destination mapping in a non-production environment. Only then decide which records can move forward automatically and which need correction.
Keep extraction separate from payment authorization. Reading an invoice is one task; deciding whether to pay it is another business process.
Lesscode’s services include document processing and system integrations. The useful outcome is a dependable record entering the right workflow—not simply a faster PDF demo. Judge the extractor by field quality, completeness and the work that remains after it runs.