# CC-OCR V2: Fine-Grained Attribution of LMM Failures in Real-World Visual Document Understanding

arXiv: https://arxiv.org/abs/2605.03903 · HTML: https://mlsanity.com/paper/2605.03903/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 27 of its 53 references that are in the map.
- First posted on arXiv 5 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 27 of 27)

- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [LayoutLMv3: Pre-training for Document AI with Unified Text and Image Masking](https://mlsanity.com/paper/2204.08387/) (2022)
- [Qwen3-VL-Embedding and Qwen3-VL-Reranker: A Unified Framework for State-of-the-Art Multimodal Retrieval and Ranking](https://mlsanity.com/paper/2601.04720/) (2026)
- [MiniCPM-V 4.5: Cooking Efficient MLLMs via Architecture, Data, and Training Recipe](https://mlsanity.com/paper/2509.18154/) (2025)
- [TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document](https://mlsanity.com/paper/2403.04473/) (2024)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models](https://mlsanity.com/paper/2502.18443/) (2025)
- [KOSMOS-2.5: A Multimodal Literate Model](https://mlsanity.com/paper/2309.11419/) (2023)
- [CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacy](https://mlsanity.com/paper/2412.02210/) (2024)
- [VRAG-RL: Empower Vision-Perception-Based RAG for Visually Rich Information Understanding via Iterative Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2505.22019/) (2025)
- [Document AI: Benchmarks, Models and Applications](https://mlsanity.com/paper/2111.08609/) (2021)
- [VRDU: A Benchmark for Visually-rich Document Understanding](https://mlsanity.com/paper/2211.15421/) (2022)
- [DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming](https://mlsanity.com/paper/2406.19101/) (2024)
- [READoc: A Unified Benchmark for Realistic Document Structured Extraction](https://mlsanity.com/paper/2409.05137/) (2024)
- [Hierarchical Visual Feature Aggregation for OCR-Free Document Understanding](https://mlsanity.com/paper/2411.05254/) (2024)
- [Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wild](https://mlsanity.com/paper/2603.04205/) (2026)
- [DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding](https://mlsanity.com/paper/2508.07313/) (2025)
- [Document Intelligence in the Era of Large Language Models: A Survey](https://mlsanity.com/paper/2510.13366/) (2025)
- [Lang2Act: Fine-Grained Visual Reasoning through Self-Emergent Linguistic Toolchains](https://mlsanity.com/paper/2602.13235/) (2026)
- [ReAlign: Optimizing the Visual Document Retriever with Reasoning-Guided Fine-Grained Alignment](https://mlsanity.com/paper/2604.07419/) (2026)
- [Revise: A Framework for Revising OCRed text in Practical Information Systems with Data Contamination Strategy](https://mlsanity.com/paper/2604.08115/) (2026)
- [Fetch-A-Set: A Large-Scale OCR-Free Benchmark for Historical Document Retrieval](https://mlsanity.com/paper/2406.07315/) (2024)
- [A Survey of OCR Evaluation Methods and Metrics and the Invisibility of Historical Documents](https://mlsanity.com/paper/2603.25761/) (2026)
- [Mitigating Judgment Preference Bias in Large Language Models through Group-Based Polling](https://mlsanity.com/paper/2510.08145/) (2025)

## Built on by (most cited first, 3 of 3)

- [How Far Is Document Parsing from Solved? PureDocBench: A Source-Traceable Benchmark across Clean, Degraded, and Real-World Settings](https://mlsanity.com/paper/2605.07492/) (2026)
- [BEAR-Bench: A Bilingual Enterprise and Academic Reasoning Benchmark for Multimodal Models](https://mlsanity.com/paper/2608.17895/) (2026)
- [EHR-Bench: A Chinese Multi-Task Benchmark for Recruitment AI](https://mlsanity.com/paper/2610.16689/) (2026)

## Abstract (arXiv)

Recent Large Multimodal Models (LMMs) have achieved remarkable progress on OCR-centric document understanding and processing tasks. Existing benchmarks primarily evaluate LMMs across diverse tasks to reflect practical document-processing workflows or analyze how document characteristics influence model performance. However, they provide limited insight into the reliability of LMMs under real-world document acquisition conditions, where factors such as lighting, screen displays, imaging quality, and capture methods can substantially affect performance. To bridge this gap, we present CC-OCR v2, a comprehensive benchmark for attributing LMM failures in real-world document processing. CC-OCR v2 provides a unified evaluation framework covering five core document-processing capabilities through 16 subtasks, 74 application scenarios, and 7,093 samples. The benchmark spans five evaluation tracks, ten document categories, and 32 languages in the recognition suite. Beyond task-level evaluation, each sample is annotated with ten fine-grained document factors, enabling systematic attribution of model failures across document types and acquisition conditions. Extensive experiments on 17 representative LMMs reveal substantial performance variation across tasks, document categories, and real-world conditions. Moreover, models with comparable overall accuracy often exhibit fundamentally different failure patterns under specific document factors, highlighting a significant gap between benchmark-level performance and reliable deployment in practical applications. The dataset and evaluation toolkit are publicly available at https://github.com/eioss/CC-OCR-V2.