mlsanityData to 14 Oct 2026

olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models

arXiv 2502.18443 ↗First posted 25 Feb 2025cs.CLcited by 125

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

PDF documents have the potential to provide trillions of novel, high-quality tokens for training language models. However, these documents come in a diversity of types with differing formats and visual layouts that pose a challenge when attempting to extract and faithfully represent the underlying content for language model use. Traditional open source tools often produce lower quality extractions compared to vision language models (VLMs), but reliance on the best VLMs can be prohibitively costly (e.g., over 6,240 USD per million PDF pages for GPT-4o) or infeasible if the PDFs cannot be sent to proprietary APIs. We present olmOCR, an open-source toolkit for processing PDFs into clean, linearized plain text in natural reading order while preserving structured content like sections, tables, lists, equations, and more. Our toolkit runs a fine-tuned 7B vision language model (VLM) trained on olmOCR-mix-0225, a sample of 260,000 pages from over 100,000 crawled PDFs with diverse properties, including graphics, handwritten text and poor quality scans. olmOCR is optimized for large-scale batch processing, able to scale flexibly to different hardware setups and can convert a million PDF pages for only 176 USD. To aid comparison with existing systems, we also introduce olmOCR-Bench, a curated set of 1,400 PDFs capturing many content types that remain challenging even for the best tools and VLMs, including formulas, tables, tiny fonts, old scans, and more. We find olmOCR outperforms even top VLMs including GPT-4o, Gemini Flash 2 and Qwen-2.5-VL. We openly release all components of olmOCR: our fine-tuned VLM model, training code and data, an efficient inference pipeline that supports vLLM and SGLang backends, and benchmark olmOCR-Bench.

Key facts. Cited by 125 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 33 of its 64 references that are in the map. First posted on arXiv 25 Feb 2025.

Builds on

33 of 64 references.

References · 33

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2025Qwen2.5-VL Technical Reportcited by 5,587
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  8. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,160
  9. 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 903
  11. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  12. 2023VILA: On Pre-training for Visual Language Modelscited by 692
21 more
  1. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  2. 2024Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Researchcited by 483
  3. 2021OCR-free Document Understanding Transformercited by 407
  4. 2024DataComp-LM: In search of the next generation of training sets for language modelscited by 398
  5. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
  6. 20242 OLMo 2 Furiouscited by 340
  7. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  8. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
  9. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  10. 2019Image-based table recognition: data, model, and evaluationcited by 237
  11. 2015Evaluation of Deep Convolutional Nets for Document Image Classification and Retrievalcited by 213
  12. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  13. 2024RegMix: Data Mixture as Regression for Language Model Pre-trainingcited by 177
  14. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Textcited by 155
  15. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 82
  16. 2025Organize the Web: Constructing Domains Enhances Pre-Training Data Curationcited by 76
  17. 2024Does your data spark joy? Performance gains from domain upsampling at the end of trainingcited by 52
  18. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  19. 2021VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groupscited by 27
  20. 2023The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfacescited by 9
  21. 20211st Place Solution for ICDAR 2021 Competition on Mathematical Formula Detectioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 37; 2026: 88.

125 papers.

Sort:

By area:

  1. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  2. 2025PaddleOCR 3.0 Technical Reportcited by 143
  3. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 116
  4. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  5. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  6. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 79
  7. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  8. 2025MegaScience: Pushing the Frontiers of Post-Training Datasets for Science Reasoningcited by 59
  9. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
  10. 2025FineVision: Open Data Is All You Needcited by 53
  11. 2025olmOCR 2: Unit Test Rewards for Document OCRcited by 47
  12. 2025POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversioncited by 34
38 more
  1. 2026STEP3-VL-10B Technical Reportcited by 32
  2. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
  3. 2026GLM-OCR Technical Reportcited by 26
  4. 2026LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCRcited by 23
  5. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 22
  6. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 22
  7. 2025UNIDOC-BENCH: A Unified Benchmark for Document-Centric Multimodal RAGcited by 21
  8. 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligencecited by 21
  9. 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Trainingcited by 21
  10. 2026AI-Assisted Peer Review at Scale: The AAAI-26 AI Review Pilotcited by 19
  11. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsingcited by 17
  12. 2025FlexOlmo: Open Language Models for Flexible Data Usecited by 15
  13. 2026Multimodal OCR: Parse Anything from Documentscited by 14
  14. 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
  15. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  16. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  17. 2026FireRed-OCR Technical Reportcited by 13
  18. 2025Consensus Entropy: Harnessing Multi-VLM Agreement for Self-Verifying and Self-Improving OCRcited by 12
  19. 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
  20. 2026Real5-OmniDocBench: A Full-Scale Physical Reconstruction Benchmark for Robust Document Parsing in the Wildcited by 12
  21. 2026Arcee Trinity Large Technical Reportcited by 11
  22. 2026ParseBench: A Document Parsing Benchmark for AI Agentscited by 11
  23. 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Modelcited by 10
  24. 2026Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decodingcited by 10
  25. 2025CHURRO: Making History Readable with an Open-Weight Large Vision-Language Model for High-Accuracy, Low-Cost Historical Text Recognitioncited by 9
  26. 2025Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCRcited by 9
  27. 2025Frustratingly Simple Retrieval Improves Challenging, Reasoning-Intensive Benchmarkscited by 8
  28. 2026MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decodingcited by 8
  29. 2026Unlimited OCR Workscited by 8
  30. 2026Data Darwinism Part I: Unlocking the Value of Scientific Data for Pre-trainingcited by 7
  31. 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
  32. 2025Datasheets Aren't Enough: DataRubrics for Automated Quality Metrics and Accountabilitycited by 6
  33. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decodingcited by 6
  34. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
  35. 2025SCAN: Semantic Document Layout Analysis for Textual and Visual Retrieval-Augmented Generationcited by 5
  36. 2026Logics-Parsing-Omni Technical Reportcited by 5
  37. 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 5
  38. 2026Reading or Guessing? Visual Grounding Failures of Vision-Language Models for OCR in Ancient Greek Editionscited by 5

50 most cited of 125 All 125 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Beyond Accuracy: Robustness, Cost, and Governance Trade-offs for Vision-Language Models in Templated Document Extraction
  2. 2026Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application
  3. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model
  4. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations
  5. 2024Building and better understanding vision-language models: insights and future directions
  6. 2024DocVLM: Make Your VLM an Efficient Reader
  7. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models
  8. 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing