mlsanityData to 14 Oct 2026

Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting

arXiv 2505.14059 ↗First posted 20 May 2025cs.CVcited by 79

Not on the map. Why?

Abstract (arXiv)

Document image parsing is challenging due to its complexly intertwined elements such as text paragraphs, figures, formulas, and tables. Current approaches either assemble specialized expert models or directly generate page-level content autoregressively, facing integration overhead, efficiency bottlenecks, and layout structure degradation despite their decent performance. To address these limitations, we present textitDolphin (textitDocument Image Parsing via Heterogeneous Anchor Prompting), a novel multimodal document image parsing model following an analyze-then-parse paradigm. In the first stage, Dolphin generates a sequence of layout elements in reading order. These heterogeneous elements, serving as anchors and coupled with task-specific prompts, are fed back to Dolphin for parallel content parsing in the second stage. To train Dolphin, we construct a large-scale dataset of over 30 million samples, covering multi-granularity parsing tasks. Through comprehensive evaluations on both prevalent benchmarks and self-constructed ones, Dolphin achieves state-of-the-art performance across diverse page-level and element-level settings, while ensuring superior efficiency through its lightweight architecture and parallel parsing mechanism. The code and pre-trained models are publicly available at https://github.com/ByteDance/Dolphin

Key facts. Cited by 79 papers in the map (data through 14 Oct 2026). Builds on 42 of its 54 references that are in the map. First posted on arXiv 20 May 2025.

Builds on

42 of 54 references.

References · 42

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2025Qwen2.5-VL Technical Reportcited by 5,587
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  7. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  8. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  9. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  10. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  11. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 681
  12. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
30 more
  1. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  2. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  3. 2020TAPAS: Weakly Supervised Table Parsing via Pre-trainingcited by 434
  4. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  5. 2023Monkey: Image Resolution and Text Label Are Important Things for Large Multi-modal Modelscited by 357
  6. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  7. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  8. 2019Image-based table recognition: data, model, and evaluationcited by 237
  9. 2024General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Modelcited by 180
  10. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Documentcited by 174
  11. 2023mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understandingcited by 168
  12. 2022Unifying Vision, Text, and Layout for Universal Document Processingcited by 140
  13. 2021PubTables-1M: Towards comprehensive table extraction from unstructured documentscited by 126
  14. 2025olmOCR: Unlocking Trillions of Tokens in PDFs with Vision Language Modelscited by 125
  15. 2023DocLLM: A layout-aware generative language model for multimodal document understandingcited by 116
  16. 2024mPLUG-DocOwl2: High-resolution Compressing for OCR-free Multi-page Document Understandingcited by 114
  17. 2023DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understandingcited by 106
  18. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  19. 2023UniDoc: A Universal Large Multimodal Model for Simultaneous Text Detection, Recognition, Spotting and Understandingcited by 74
  20. 2024TabPedia: Towards Comprehensive Visual Table Understanding with Concept Synergycited by 71
  21. 2024TextSquare: Scaling up Text-Centric Visual Instruction Tuningcited by 62
  22. 2024Focus Anywhere for Fine-grained Multi-page Document Understandingcited by 62
  23. 2023Multi-modal In-Context Learning Makes an Ego-evolving Scene Text Recognizercited by 53
  24. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  25. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
  26. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  27. 2023Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMscited by 25
  28. 2022Wukong-Reader: Multi-modal Pre-training for Fine-grained Visual Document Understandingcited by 20
  29. 2022On Web-based Visual Corpus Construction for Visual Document Understandingcited by 5
  30. 2024DocFusion: A Unified Framework for Document Parsing Taskscited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 60.

79 papers.

Sort:

By area:

  1. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 183
  2. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  3. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  4. 2026DeepSeek-OCR 2: Visual Causal Flowcited by 74
  5. 2026PaddleOCR-VL-1.5: Towards a Multi-Task 0.9B VLM for Robust In-the-Wild Document Parsingcited by 57
  6. 2025dots.ocr: Multilingual Document Layout Parsing in a Single Vision-Language Modelcited by 47
  7. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
  8. 2026GLM-OCR Technical Reportcited by 26
  9. 2026Dolphin-v2: Universal Document Parsing via Scalable Anchor Promptingcited by 22
  10. 2026PaddleOCR-VL-1.6: Expanding the Frontier of Document Parsing with Under-Optimized Region Refinement and Progressive Post-Trainingcited by 21
  11. 2025HunyuanOCR Technical Reportcited by 20
  12. 2025Logics-Parsing Technical Reportcited by 17
38 more
  1. 2026ClawGUI: A Unified Framework for Training, Evaluating, and Deploying GUI Agentscited by 14
  2. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  3. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  4. 2026FireRed-OCR Technical Reportcited by 13
  5. 2026OCRVerse: Towards Holistic OCR in End-to-End Vision-Language Modelscited by 12
  6. 2026Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decodingcited by 10
  7. 2025DocPTBench: Benchmarking End-to-End Photographed Document Parsing and Translationcited by 9
  8. 2025Reading or Reasoning? Format Decoupled Reinforcement Learning for Document OCRcited by 9
  9. 2026Unlimited OCR Workscited by 8
  10. 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
  11. 2025Advanced Layout Analysis Models for Doclingcited by 6
  12. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  13. 2026HSD: Training-Free Acceleration for Document Parsing Vision-Language Models with Hierarchical Speculative Decodingcited by 6
  14. 2026DocSeeker: Structured Visual Reasoning with Evidence Grounding for Long Document Understandingcited by 6
  15. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
  16. 2026AgenticOCR: Parsing Only What You Need for Efficient Retrieval-Augmented Generationcited by 5
  17. 2026Logics-Parsing-Omni Technical Reportcited by 5
  18. 2026OvisOCR2 Technical Reportcited by 5
  19. 2025TRivia: Self-supervised Fine-tuning of Vision-Language Models for Table Recognitioncited by 4
  20. 2026Beyond the Grid: Layout-Informed Multi-Vector Retrieval with Parsed Visual Document Representationscited by 4
  21. 2025Uni-Parser Technical Reportcited by 3
  22. 2026Boosting Document Parsing Efficiency and Performance with Coarse-to-Fine Visual Processingcited by 3
  23. 2026ABot-OCR Technical Reportcited by 3
  24. 2025Complex Mathematical Expression Recognition: Benchmark, Large-Scale Dataset and Strong Baselinecited by 2
  25. 2025DAVE: A VLM Vision Encoder for Document Understanding and Web Agentscited by 2
  26. 2026LingDT-VL-OCR: Structure-Aware Document-Level Parsing with Fine-Grained Visual Referencecited by 2
  27. 2026Efficient Document Parsing via Parallel Token Predictioncited by 2
  28. 2026SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asiacited by 2
  29. 2026RT-DocLayout: Real-Time End-to-End Document Layout Analysis with Reading Order in the Wildcited by 2
  30. 2026P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scalingcited by 2
  31. 2026MORE: A Multilingual Document Parsing Benchmark and Evaluationcited by 2
  32. 2025SciPostLayoutTree: A Dataset for Structural Analysis of Scientific Posterscited by 1
  33. 2025NVIDIA Nemotron Parse 1.1cited by 1
  34. 2026TDATR: Improving End-to-End Table Recognition via Table Detail-Aware Learning and Cell-Level Visual Alignmentcited by 1
  35. 2026DocAtlas: Multilingual Document Understanding Across 80+ Languagescited by 1
  36. 2026PaDoc: Layout-Grounded Parallel Decoding for Document Parsingcited by 1
  37. 2025Cascaded Robust Rectification for Arbitrary Document Imagescited by 0
  38. 2026LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Datacited by 0

50 most cited of 79 All 79 (JSON)