A vision-language model does not process an image the way a classifier does. An encoder converts the image into a sequence of embeddings, those embeddings are projected into the same space the language model uses for text tokens, and the language model then reads them as though they were words. Everything the model can do with an image follows from that: it is reading a translated description produced by the encoder, at whatever fidelity the encoder preserved.
That framing explains most of the strengths and all of the limitations.
What changed in 2026
- High-resolution handling improved substantially. Tiling approaches that split a large image into patches processed at native resolution largely replaced naive downscaling, which is why reading small text got dramatically better.
- Document understanding became a headline use. Vision models became a credible alternative to traditional extraction pipelines for invoices, forms, and reports.
- Video support extended the pattern. Frame sampling with temporal context moved from research into production APIs, with the token costs that implies.
- Cost awareness increased. As image inputs became routine, teams noticed that images dominate their token bills and started optimizing what they send.
What they are good and bad at
| Task |
Reliability |
| Describing scene content |
High |
| Reading printed text in an image |
High, with adequate resolution |
| Understanding chart trends |
Good; exact values less so |
| Reading tables from a page image |
Good, and better than most parsers |
| Counting many similar objects |
Poor |
| Precise coordinates or bounding boxes |
Poor to moderate |
| Fine spatial relations |
Moderate; left and right errors are common |
| Reading handwriting |
Variable; depends heavily on legibility |
| Detecting subtle image manipulation |
Poor |
The counting and coordinates rows surprise people. A model that describes a complex photograph fluently will miscount fifteen items in it, because the encoded representation preserves semantics far better than precise quantity or position. If your task needs bounding boxes, a purpose-built detection model remains the better tool.
Managing the cost
Images are expensive. The token cost of an image scales with resolution, and a high-resolution page tiled for detail can consume more input tokens than several thousand words of text. At volume this dominates the bill.
Three practical reductions. Crop to the region of interest before sending — most of a photograph is usually irrelevant to the question. Downscale when detail is not needed, since a model asked to describe a scene does not need the resolution required to read eight-point footnotes. And resize deliberately rather than sending whatever the camera produced, because a phone photo is far larger than any model needs for most tasks.
Test where your resolution floor actually is. There is usually a size below which accuracy on your specific task drops sharply and above which it does not improve. Finding that point once can cut image spend substantially, and it interacts directly with the accuracy measurement in AI document extraction accuracy.
Common mistakes
- Sending full-resolution originals. Cost scales with pixels; most of them are not helping.
- Expecting reliable counting. Use a detection model when quantity matters.
- Assuming reading text means understanding layout. Table structure is better than it used to be and still worth verifying.
- Skipping resolution testing. Both over-sending and under-sending are common and both are costly, in money or accuracy.
- Using a vision model for image forensics. Detecting manipulation is a different problem requiring different tools.
FAQ
Are vision models better than traditional OCR?
For layout-heavy and messy documents, generally yes, because they read structure rather than emitting a flat character stream. For high-volume clean text, dedicated OCR is faster and cheaper.
How many images can I send in one request?
Provider limits vary, and the practical limit is usually context window rather than a hard count. Several high-resolution images fill a window quickly.
Do they work on video?
Through frame sampling, yes. Token cost scales with the number of frames, so sampling strategy is the main design decision.
Can they read handwriting?
Better than traditional OCR in most cases, with accuracy that varies widely by legibility. Test on your actual samples rather than assuming.
Where to go next
For document pipelines, read AI document extraction accuracy and multimodal RAG explained. For managing the token spend, AI cost optimization.