MORE: A Multilingual Document Parsing Benchmark and Evaluation
Not on the map. Why?
Abstract (arXiv)
Multilingual documents encapsulate rich regional cultures, scientific discoveries, and historical records. Parsing this content into structured, machine-readable formats is critical for unlocking global knowledge. However, existing benchmarks predominantly focus on high-resource languages like English and Chinese, creating an evaluation blind spot concerning model performance on other languages. While recent Vision-Language Models (VLMs) claim support for hundreds of languages, the lack of ground truth makes it impossible to empirically verify these capabilities. To bridge this gap, we introduce MORE, a large-scale benchmark designed for multilingual document parsing evaluation. MORE distinguishes itself through three key dimensions: (1) Unprecedented Scale: It covers 149 languages, making it the most linguistically diverse benchmark to date; (2) Structural Complexity: Unlike previous works, it extends evaluation beyond plain text to include structural elements such as code blocks, tables, and catalogs; and (3) Data Authenticity: All samples are curated from real-world documents via a model-assisted, human-refined annotation pipeline. We evaluate state-of-the-art models using MORE, establishing new performance baselines for long-tail languages and validating the benchmark's effectiveness in diagnosing model capabilities in realistic, diverse scenarios. The MORE dataset will be available at https://github.com/zimoqingfeng/MORE.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 3 Jul 2026.
Builds on
30 of 35 references.
References · 30
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
- 2021OCR-free Document Understanding Transformercited by 407
- 2025Seed1.5-VL Technical Reportcited by 373
- 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
- 2019Image-based table recognition: data, model, and evaluationcited by 237
18 more
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
- 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
- 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
- 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
- 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
- 2024CC-OCR: A Comprehensive and Challenging OCR Benchmark for Evaluating Large Multimodal Models in Literacycited by 81
- 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 79
- 2024D-FINE: Redefine Regression Task in DETRs as Fine-grained Distribution Refinementcited by 74
- 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
- 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
- 2023CCpdf: Building a High Quality Corpus for Visually Rich Documents from Web Crawl Datacited by 21
- 2025HunyuanOCR Technical Reportcited by 20
- 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
- 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 5
- 2026Chronicles-OCR: A Cross-Temporal Perception Benchmark for the Evolutionary Trajectory of Chinese Characterscited by 5
- 2025Beyond Cropped Regions: New Benchmark and Corresponding Baseline for Chinese Scene Text Retrieval in Diverse Layoutscited by 4
- 2026MMTIT-Bench: A Multilingual and Multi-Scenario Benchmark with Cognition-Perception-Reasoning Guided Text-Image Machine Translationcited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026DocAtlas: Multilingual Document Understanding Across 80+ Languages
- 2026SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asia
- 2026FineWeb-CLaR: Culture, Language, and Region Annotations for Benchmark-Aligned Corpus Auditing
- 2026Retrieval or Representation? Reassessing Benchmark Gaps in Multilingual and Visually Rich RAG
- 2024Multilingual Large Language Model: A Survey of Resources, Taxonomy and Frontiers
- 2024A Survey on Multilingual Large Language Models: Corpora, Alignment, and Bias
- 2025Are We on the Right Way for Assessing Document Retrieval-Augmented Generation?
- 2025VisR-Bench: An Empirical Study on Visual Retrieval-Augmented Generation for Multilingual Long Document Understanding