OCR reached the point where reading characters off a reasonably clean page is largely solved. Accuracy on printed text is high enough that transcription errors are no longer the main obstacle.
What remains hard is everything after that. You have a page of words with positions. You need to know that this number is the total and that one is the tax, that these rows form a table, and that this date is the invoice date rather than the due date.
What changed in 2026
- Multimodal models entered document processing. Vision-language models handling layout directly reduced reliance on separate structure detection.
- Hybrid pipelines became standard. OCR for exact characters plus a model for structure and meaning outperformed either alone.
- Table extraction stayed the weak point. Complex tables with merged cells and spanning headers remained the common failure.
- Verification against source became expected. Extracted values traceable to a location on the page became a standard requirement in regulated settings.
Skip OCR when you can
The first question is whether you need OCR at all.
A born-digital PDF — generated from a document rather than scanned — contains the actual text. Extracting it directly is exact, fast, and free of transcription error. Running OCR on a rendering of it introduces errors that were not there.
The catch is that some born-digital PDFs have poor internal text ordering, so extracted text can be jumbled even though every character is correct. Checking whether extracted text is sensible, and falling back to OCR only when it is not, is a reasonable pipeline.
Scanned documents and photographs genuinely need OCR. So do PDFs containing scanned page images, which look identical to born-digital ones until you try to extract from them.
Structure is the hard part
| Task |
Difficulty |
Notes |
| Reading printed characters |
Low |
Largely solved |
| Reading handwriting |
Moderate to high |
Varies enormously |
| Reading order on a page |
Moderate |
Columns and sidebars confuse it |
| Associating labels with values |
Moderate |
Where most extraction errors live |
| Simple tables |
Moderate |
Usually workable |
| Complex tables |
High |
Merged cells, spanning headers |
| Multi-page tables |
High |
Continuation and repeated headers |
Label-value association is where most practical errors occur. A page contains several dates and several amounts; knowing which is the invoice date and which is the total requires understanding layout, not just reading.
This is where a model helps genuinely — given OCR output with positions, it can reason about which label a value belongs to far better than a rule-based approach. The division of labour that works: OCR provides exact characters, the model provides interpretation.
Keep the positions
The single most valuable thing to preserve through a pipeline is bounding boxes — where on the page each piece of text was found.
They enable three things that are otherwise impossible.
Structure reconstruction. Column alignment, row grouping, and reading order all follow from geometry.
Verification. A value traceable to a specific location can be checked by a human looking at the original, and highlighted for review. Extraction without provenance is unverifiable.
Confidence signals. Values found in unexpected locations, or far from their expected label, are candidates for review.
Pipelines that discard positions and pass plain text to a model lose all of this, and it is the difference between a system you can audit and one you have to trust.
Common mistakes
- Running OCR on born-digital PDFs. Introduces errors that were not present.
- Discarding bounding boxes. Loses structure, verification, and confidence.
- Assuming character accuracy solves extraction. Structure is the harder half.
- Naive table parsing. Merged cells and spanning headers break it.
- No validation of extracted values. A plausible wrong total is worse than a failure.
- Ignoring page orientation and skew. Preprocessing matters more than people expect.
- Treating handwriting like print. Very different accuracy expectations.
FAQ
Should I use OCR or a vision model?
Both, for different parts. OCR for exact characters, which vision models read less reliably; the model for layout, structure, and meaning. Choosing one alone gives up the other's strength — see vision encoders.
How do I handle tables?
With difficulty, and by preserving geometry. Complex tables warrant a dedicated table-detection step, and validating that extracted rows sum correctly catches many errors, per LLM output validation.
What accuracy should I expect?
High on clean printed text, substantially lower on handwriting, poor scans, and unusual layouts. Measure on your actual documents rather than on a benchmark.
How do I validate extracted fields?
Check them against the source geometrically, cross-check totals against line items, and validate formats. Anything you act on needs a check that does not come from the same model.
Where to go next
For how vision models read images, read vision encoders. For validating what comes out, LLM output validation, and for indexing documents afterwards, contextual retrieval.