# ExtractBench: A Benchmark for Schema-Guided Enterprise Document Extraction

arXiv: https://arxiv.org/abs/2607.29677 · HTML: https://mlsanity.com/paper/2607.29677/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 12 of 12)

- [FUNSD: A Dataset for Form Understanding in Noisy Scanned Documents](https://mlsanity.com/paper/1905.13538/) (2019)
- [CUAD: An Expert-Annotated NLP Dataset for Legal Contract Review](https://mlsanity.com/paper/2103.06268/) (2021)
- [Kleister: Key Information Extraction Datasets Involving Long Documents with Complex Layouts](https://mlsanity.com/paper/2105.05796/) (2021)
- [DocILE Benchmark for Document Information Localization and Extraction](https://mlsanity.com/paper/2302.05658/) (2023)
- [VRDU: A Benchmark for Visually-rich Document Understanding](https://mlsanity.com/paper/2211.15421/) (2022)
- [ExtractBench: A Benchmark and Evaluation Methodology for Complex Structured Extraction](https://mlsanity.com/paper/2602.12247/) (2026)
- [Visual Template Inference for Data Extraction from Documents](https://mlsanity.com/paper/2501.06659/) (2025)
- [Document Haystack: A Long Context Multimodal Image/Document Understanding Vision LLM Benchmark](https://mlsanity.com/paper/2507.15882/) (2025)
- [UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents](https://mlsanity.com/paper/2602.07038/) (2026)
- [VAREX: A Benchmark for Multi-Modal Structured Extraction from Documents](https://mlsanity.com/paper/2603.15118/) (2026)
- [FormGym: Doing Paperwork with Agents](https://mlsanity.com/paper/2506.14079/) (2025)
- [ExStrucTiny: A Benchmark for Schema-Variable Structured Information Extraction from Document Images](https://mlsanity.com/paper/2602.12203/) (2026)

## Built on by (most cited first, 1 of 1)

- [The Last Mile of Coding Agents for Long-Document Tasks: An Empirical Study on Clinical Trial Protocol Digitization](https://mlsanity.com/paper/2610.18507/) (2026)

## Abstract (arXiv)

Enterprise workflows increasingly rely on agents for schema-guided extraction: given a document and a user-defined schema, the agent faithfully follows the schema to produce the correct output with source evidence as grounding metadata. We present ExtractBench, a benchmark for schema-guided extraction and, to our knowledge, the first to score value accuracy, record completeness at scale, grounding, and measured cost together. The evaluation system contains 4,869 pages across 370 enterprise documents, 8 business domains, and 67 document types, with clear tags differentiating their challenge scenarios. The scalable schema and ground-truth curation pipeline combines independent-system agreement for real documents, known values for synthetic lists, and human verification for forms. We report order-insensitive value F1 for value accuracy, plus two grounding metrics for source traceability: word- and page-level F1. Commercial VLMs perform well on short documents but often truncate record lists on long ones, while coding agents retain higher accuracy at much higher cost. LlamaExtract Agentic Plus ranks first on all three metrics, with accuracy comparable to coding agents at a fraction of the cost. Dataset and evaluation code are available on hrefhttps://huggingface.co/datasets/llamaindex/ExtractBenchHuggingFace and hrefhttps://github.com/run-llama/ExtractBenchGitHub.