# ParseBench: A Document Parsing Benchmark for AI Agents

arXiv: https://arxiv.org/abs/2604.08538 · HTML: https://mlsanity.com/paper/2604.08538/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Web, search, and research agents, one of 16 directions in LLM agents and tool use.
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 9 Apr 2026.

## Placed in

- LLM agents and tool use / [Web, search, and research agents](https://mlsanity.com/agents/web-search-and-research-agents/)

## Builds on (most cited first, 12 of 12)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [OpenAI GPT-5 System Card](https://mlsanity.com/paper/2601.03267/) (2025)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [OCRBench v2: An Improved Benchmark for Evaluating Large Multimodal Models on Visual Text Localization and Reasoning](https://mlsanity.com/paper/2501.00321/) (2024)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis](https://mlsanity.com/paper/2206.01062/) (2022)
- [Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Context](https://mlsanity.com/paper/2005.00589/) (2020)
- [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025)
- [Docling: An Efficient Open-Source Toolkit for AI-driven Document Conversion](https://mlsanity.com/paper/2501.17887/) (2025)
- [GriTS: Grid table similarity metric for table structure recognition](https://mlsanity.com/paper/2203.12555/) (2022)

## Built on by (most cited first, 11 of 11)

- [How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings](https://mlsanity.com/paper/2605.07492/) (2026)
- [ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formats](https://mlsanity.com/paper/2606.01348/) (2026)
- [MADP: A Multi-Agent Pipeline for Sustainable Document Processing with Human-in-the-Loop](https://mlsanity.com/paper/2605.17159/) (2026)
- [MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing](https://mlsanity.com/paper/2605.22100/) (2026)
- [WorldBench: Culturally Grounded Benchmark for Multilingual Agents](https://mlsanity.com/paper/2609.01056/) (2026)
- [EpiCon: Collective Agent Learning through Co-Evolving Multimodal Memory](https://mlsanity.com/paper/2609.37923/) (2026)
- [RealDocBench: A Benchmark for Field-Level QA and Layout Understanding on Real-World Regulated Documents](https://mlsanity.com/paper/2606.07401/) (2026)
- [IDP AutoOpt: Agent-Driven Optimization of Document Processing Pipeline Configurations](https://mlsanity.com/paper/2607.26075/) (2026)
- [Chartography: A Benchmark for Professional Chart Understanding](https://mlsanity.com/paper/2608.10677/) (2026)
- [StagedWorkspace: A Versioned Workspace for Knowledge-Work Agents](https://mlsanity.com/paper/2608.18050/) (2026)
- [A Strong Harness Narrows the Model Gap: A Comparative Evaluation for Legislative Document-to-XML Conversion](https://mlsanity.com/paper/2610.19999/) (2026)

## Abstract (arXiv)

AI agents are changing the requirements for document parsing. What matters is semantic correctness: parsed output must preserve the structure and meaning needed for autonomous decisions, including correct table structure, precise chart data, semantically meaningful formatting, and visual grounding. Existing benchmarks do not fully capture this setting for enterprise automation, relying on narrow document distributions and text-similarity metrics that miss agent-critical failures. We introduce ParseBench, a benchmark of ~2,000 human-verified pages from enterprise documents spanning insurance, finance, and government, organized around five capability dimensions: tables, charts, content faithfulness, semantic formatting, and visual grounding. Across 14 methods spanning vision-language models, specialized document parsers, and LlamaParse, the benchmark reveals a fragmented capability landscape: no method is consistently strong across all five dimensions. LlamaParse Agentic achieves the highest overall score at 84.9%, and the benchmark highlights the remaining capability gaps across current systems. Dataset and evaluation code are available on https://huggingface.co/datasets/llamaindex/ParseBench and https://github.com/run-llama/ParseBench.