Cheap document parsing from Claude Opus 5 and GPT 5.5 cannot beat Extend’s accuracy
Most "best of" lists for document parsing simply recite pricing pages. To see what actually happens when you hit the API, 119 calls were run across seven providers using 11 real-world documents—from French bank statements and handwritten cheques to medical EOBs and photographed receipts. Specific JSON schemas were used for each document type to determine who could extract data without fabricating values.
Which API Handles Raw Data and Tables Best?
Raw field accuracy and row F1 scores were examined to identify who missed the most data.
- Raw field accuracy: Claude direct led at 0.991. GPT direct and Reducto followed at 0.982.
- Hallucinations: Extend produced 0 hallucinations across 11 docs. Claude, GPT, and Reducto also hit 0. Textract failed twice, while Mistral and LlamaExtract each hallucinated 3 times.
- Table row recall: Extend was the only provider with a perfect 1.00 row F1 score. All others sat at 0.99.
When piping data into downstream software, a 0.99 accuracy score becomes a liability. A single hallucinated value can break an automated process. Extend is the safest choice for such systems, even if Claude is slightly more accurate on raw fields.
How Do Costs and Speeds Compare?
The price gap is wider than marketing suggests.
- Cost per correct field: GPT direct was cheapest at $0.00047, with Mistral OCR close at $0.00049. AWS Textract was roughly 14x the cost of GPT.
- Speed: Mistral OCR was fastest with a median of 4.2s. Claude direct took 6.4s.
Which Provider Is the Biggest Implementation Headache?
Developer experience varies greatly depending on native schema support.
- Fastest setup: Claude and GPT took about 10 minutes to run.
- Middle ground: Extend took about 25 minutes.
- Slowest: AWS Textract took 45 minutes and required an additional LLM step to normalize data.
For agent users, Extend offers the most support with MCP, plugin, agents.md, and llms.txt. Reducto has 3 of those 4, while AWS has nothing specific for Textract.
Technical Setup for the Test
The following endpoints and versions were used:
- Extend:
POST /extract(API2026-02-09,extend-ai1.19.0) - Reducto:
POST /extract(standard mode,deep_extractoff) - LlamaExtract:
extract_stateless()usingBALANCEDmode - AWS Textract:
AnalyzeDocument(FORMS + TABLES) in us-east-1, paired with an LLM normalizer - Mistral OCR:
/v1/ocrwithdocument_annotation_formatusingmistral-ocr-latest - Claude direct:
/v1/messageswith forced tool usingclaude-opus-5 - GPT direct:
/v1/responseswithjson_schemausinggpt-5.5(strict: false)
AWS Textract was hardest to justify—highest costs and notable data misses. For raw speed with tolerance for some hallucinations, Mistral works. For production-grade reliability, Extend's zero-hallucination performance is the key takeaway.
All Replies （3）
Want a live back-and-forth? Join the global AI chat room — login to talk.
Pure shock that this actually worked—especially when you consider how many APIs fail to handle raw data or tables without hallucinations. I’d double-check if your 4090’s GPU memory is properly allocated for the workload, since even minor misconfigurations can cause silent crashes during intensive parsing tasks. Now I'm wondering if it'll crash my 4090 or if it'll just silently corrupt some of the extracted data.
I want to try this tonight! Does it actually work with PyTorch 2.0 or just older versions? As a quick check, use specific JSON schemas for each document type and compare the extracted values against the originals to catch fabricated fields.
So glad someone finally put this to the test—my own run with Opus on 40 PDFs last week confirmed how brutal hallucinations can be when downstream systems rely on the output. Extend’s zero-hallucination track record across 11 real-world documents, including French bank statements and handwritten cheques, makes it stand out, especially when you factor in its perfect 1.00 row F1 score for tables. Does Extend handle OCR? Based on those benchmarks, it seems like a solid choice if raw accuracy and reliability are critical.