FireRed-OCR Technical Report
Not on the map. Why?
Abstract (arXiv)
We present FireRed-OCR, a systematic framework to specialize general VLMs into high-performance OCR models. Large Vision-Language Models (VLMs) have demonstrated impressive general capabilities but frequently suffer from ``structural hallucination'' when processing complex documents, limiting their utility in industrial OCR applications. In this paper, we introduce FireRed-OCR, a novel framework designed to transform general-purpose VLMs (based on Qwen3-VL) into pixel-precise structural document parsing experts. To address the scarcity of high-quality structured data, we construct a ``Geometry + Semantics'' Data Factory. Unlike traditional random sampling, our pipeline leverages geometric feature clustering and multi-dimensional tagging to synthesize and curate a highly balanced dataset, effectively handling long-tail layouts and rare document types. Furthermore, we propose a Three-Stage Progressive Training strategy that guides the model from pixel-level perception to logical structure generation. This curriculum includes: (1) Multi-task Pre-alignment to ground the model's understanding of document structure; (2) Specialized SFT for standardizing full-image Markdown output; and (3) Format-Constrained Group Relative Policy Optimization (GRPO), which utilizes reinforcement learning to enforce strict syntactic validity and structural integrity (e.g., table closure, formula syntax). Extensive evaluations on OmniDocBench v1.5 demonstrate that FireRed-OCR achieves state-of-the-art performance with an overall score of 92.94%, significantly outperforming strong baselines such as DeepSeek-OCR 2 and OCRVerse across text, formula, table, and reading order metrics. We open-source our code and model weights to facilitate the ``General VLM to Specialized Structural Expert'' paradigm.
Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 36 of its 47 references that are in the map. First posted on arXiv 2 Mar 2026.
Builds on
36 of 47 references.
References · 36
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,769
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
- 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
24 more
- 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
- 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 255
- 2019Image-based table recognition: data, model, and evaluationcited by 233
- 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
- 2021TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene textcited by 170
- 2022VLP: A Survey on Vision-Language Pre-trainingcited by 159
- 2019ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (RRC-ArT)cited by 147
- 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 144
- 2025PaddleOCR 3.0 Technical Reportcited by 136
- 2023X-LLM: Bootstrapping Advanced Large Language Models by Treating Multi-Modalities as Foreign Languagescited by 127
- 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 120
- 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsingcited by 108
- 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 84
- 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 81
- 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 78
- 2026DeepSeek-OCR 2: Visual Causal Flowcited by 68
- 2025Meta CLIP 2: A Worldwide Scaling Recipecited by 59
- 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 54
- 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 44
- 2025POINTS-Reader: Distillation-Free Adaptation of Vision-Language Models for Document Conversioncited by 34
- 2026STEP3-VL-10B Technical Reportcited by 28
- 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
- 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 10
- 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUscited by 2
Built on by
Citing papers per year: 2026: 12.
12 papers.
- 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Bettercited by 14
- 2026Unlimited OCR Workscited by 8
- 2026OvisOCR2 Technical Reportcited by 5
- 2026ABot-OCR Technical Reportcited by 3
- 2026Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsingcited by 1
- 2026FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarkscited by 1
- 2026MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsingcited by 0
- 2026HPD-Parsing: Hierarchical Parallel Document Parsingcited by 0
- 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humanscited by 0
- 2026WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsingcited by 0
- 2026DocMIDE: Learning Multi-Hop Implicit Derivation in Visually Rich Documentscited by 0
- 2026Xiaomi-OCR-0 Technical Reportcited by 0
- 2026Xiaomi-OCR-0 Technical Report28 Sep 2026
- 2026DocMIDE: Learning Multi-Hop Implicit Derivation in Visually Rich Documents21 Sep 2026
- 2026WeVisDoc: From Coverage to Capability for Robust End-to-End Document Parsing17 Sep 2026
- 2026WildHandBench: A Benchmark for Handwritten Text Understanding that Challenges MLLMs and Humans24 Aug 2026
- 2026FinixDoc: Rethinking Financial Document Parsing Beyond Saturated Benchmarks24 Aug 2026
- 2026HPD-Parsing: Hierarchical Parallel Document Parsing21 Jul 2026
- 2026OvisOCR2 Technical Report15 Jul 2026
- 2026HunyuanOCR-1.5: Making Lightweight OCR VLMs Faster and Better6 Jul 2026
- 2026Unlimited OCR Works22 Jun 2026
- 2026ABot-OCR Technical Report27 May 2026
- 2026MPDocBench-Parse: Benchmarking Practical Multi-page Document Parsing21 May 2026
- 2026Molecular Identifier Visual Prompt and Verifiable Reinforcement Learning for Chemical Reaction Diagram Parsing16 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026Qianfan-OCR: A Unified End-to-End Model for Document Intelligence
- 2026Do VLMs Read or Rewrite? On Transcription Faithfulness in Vision-Language Models
- 2026DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth
- 2024TAP-VL: Text Layout-Aware Pre-training for Enriched Vision-Language Models
- 2026LayoutLite: Token-Level Implicit Layout Analysis for Efficient Document OCR
- 2026Evaluating Structured Information Extraction with Open Models in a High Risk Public Sector Application
- 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understanding
- 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Model