mlsanityData to 14 Oct 2026

Infinity-Parser2 Technical Report

arXiv 2607.07836 ↗First posted 8 Jul 2026cs.AIcited by 4

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

also in 1 more
Abstract (arXiv)

We present Infinity-Parser2, a large multimodal model that couples a controllable data-synthesis pipeline with multi-task reinforcement learning for end-to-end document parsing, addressing the persistent scarcity of faithfully annotated parsing corpora. Our contributions are threefold. First, we build a scalable synthesis engine, pairing a controllable rendering framework with an iterative refinement loop, and use it to construct and open-source Infinity-Doc2-5M: a 5-million-sample bilingual (Chinese/English) corpus spanning diverse document types, annotated with element bounding boxes, canonical content forms (Markdown, HTML, LaTeX, SMILES, structured charts), and full-page reading order. Second, we introduce a verifiable, multi-task reward system that enables Joint Reinforcement Learning across eight co-trained objectives (document parsing, layout analysis, table parsing, math formula parsing, chart parsing, chemical formula parsing, document VQA, and general multimodal understanding), unifying perception, structure, and reasoning in a single optimization signal. Third, we release two variants under a shared architecture: Infinity-Parser2-Flash, optimized for low-latency inference with a 3.68x throughput gain over Infinity-Parser-7B, and Infinity-Parser2-Pro, engineered for precision-critical settings. Infinity-Parser2-Pro reaches state-of-the-art 87.6% on olmOCR-Bench and 74.3% on ParseBench, surpassing DeepSeek-OCR-2, PaddleOCR-VL-1.5, and MinerU2.5, with strong generalization to charts, chemical formulas, and document VQA.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 84 references that are in the map. First posted on arXiv 8 Jul 2026.

Builds on

62 of 84 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR20 Jan
  2. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Model2 Dec
  3. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing17 Oct
  4. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Model16 Oct
  5. 2025MinerU2.5: A Decoupled Vision-Language Model for Efficient High-Resolution Document Parsing26 Sep
  6. 2025Logics-Parsing Technical Report24 Sep
  7. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Models25 Feb
  8. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models13 May

Other references · 54

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  3. 2018Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentationcited by 3,223
  4. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  5. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
  6. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  7. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  8. 2019Towards VQA Models That Can Readcited by 1,718
  9. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
38 more
  1. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,005
  2. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?cited by 914
  3. 2016A Diagram Is Worth A Dozen Imagescited by 804
  4. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 708
  5. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  6. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 355
  7. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 353
  8. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  9. 2019Image-based table recognition: data, model, and evaluationcited by 237
  10. 2023TextDiffuser: Diffusion Models as Text Painterscited by 229
  11. 2024mPLUG-DocOwl 1.5: Unified Structure Learning for OCR-free Document Understandingcited by 220
  12. 2024ALLaVA: Harnessing GPT4V-Synthesized Data for Lite Vision-Language Modelscited by 205
  13. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  14. 2023AnyText: Multilingual Visual Text Generation And Editingcited by 170
  15. 2022Chart-to-Text: A Large-Scale Benchmark for Chart Summarizationcited by 169
  16. 2022DePlot: One-shot visual language reasoning by plot-to-table translationcited by 160
  17. 2023UniChart: A Universal Vision-language Pretrained Model for Chart Comprehension and Reasoningcited by 160
  18. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
  19. 2022DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysiscited by 148
  20. 2020Global Table Extractor (GTE): A Framework for Joint Table Identification and Cell Structure Recognition Using Visual Contextcited by 147
  21. 2023To See is to Believe: Prompting GPT-4V for Better Visual Instruction Tuningcited by 144
  22. 2024ChartMimic: Evaluating LMM's Cross-Modal Reasoning Capability via Chart-to-Code Generationcited by 129
  23. 2022TableFormer: Table Structure Understanding with Transformerscited by 94
  24. 2025ChartCoder: Advancing Multimodal Large Language Model for Chart-to-Code Generationcited by 92
  25. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  26. 2024DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perceptioncited by 82
  27. 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
  28. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  29. 2025Scaling Text-Rich Image Understanding via Code-Guided Synthetic Multimodal Data Generationcited by 58
  30. 2023Vision Grid Transformer for Document Layout Analysiscited by 47
  31. 2025olmOCR 2: Unit Test Rewards for Document OCRcited by 47
  32. 2024ChartMoE: Mixture of Diversely Aligned Expert Connector for Chart Understandingcited by 46
  33. 2024Multimodal Table Understandingcited by 44
  34. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
  35. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  36. 2024OneChart: Purify the Chart Structural Extraction via One Auxiliary Tokencited by 39
  37. 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
  38. 2026GLM-OCR Technical Reportcited by 26

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Infinity Parser: Layout Aware Reinforcement Learning for Scanned Document Parsing
  2. 2025DocR1: Evidence Page-Guided GRPO for Multi-Page Document Understanding
  3. 2025Structured Document Translation via Format Reinforcement Learning
  4. 2025STRuCT-LLM: Unifying Tabular and Graph Reasoning with Reinforcement Learning for Semantic Parsing
  5. 2026DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards
  6. 2026PARSER: Read in Parallel, Reason in Depth for Long-Context LLM Agents
  7. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image
  8. 2026CSPO: Alleviating Reward Ambiguity for Structured Table-to-LaTeX Generation