mlsanityData to 14 Oct 2026

DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarios

arXiv 2508.00311 ↗First posted 1 Aug 2025cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Optical Character Recognition (OCR) for mathematical formula is essential for the intelligent analysis of scientific literature. However, both task-specific and general vision-language models often struggle to handle the structural diversity, complexity, and real-world variability inherent in mathematical content. In this work, we present DocTron-Formula, a unified framework built upon general vision-language models, thereby eliminating the need for specialized architectures. Furthermore, we introduce CSFormula, a large-scale and challenging dataset that encompasses multidisciplinary and structurally complex formulas at the line, paragraph, and page levels. Through straightforward supervised fine-tuning, our approach achieves state-of-the-art performance across a variety of styles, scientific domains, and complex layouts. Experimental results demonstrate that our method not only surpasses specialized models in terms of accuracy and robustness, but also establishes a new paradigm for the automated understanding of complex scientific documents.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

26 of 40 references.

References · 26

  1. 2017Attention Is All You Needcited by 60,361
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  6. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  7. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  8. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  9. 2021OCR-free Document Understanding Transformercited by 407
  10. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
  11. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  12. 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
14 more
  1. 2023SVIT: Scaling up Visual Instruction Tuningcited by 162
  2. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  3. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
  4. 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
  5. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
  6. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  7. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 36
  8. 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
  9. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  10. 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
  11. 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
  12. 2019Pattern Generation Strategies for Improving Recognition of Handwritten Mathematical Expressionscited by 13
  13. 2025PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognitioncited by 7
  14. 2024Enhancing Complex Formula Recognition with Hierarchical Detail-Focused Networkcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters
  2. 2024STEM-POM: Evaluating Language Models Math-Symbol Reasoning in Document Parsing
  3. 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
  4. 2024ChatSR: Multimodal Large Language Models for Scientific Formula Discovery
  5. 2024MathWriting: A Dataset For Handwritten Mathematical Expression Recognition
  6. 2026Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy
  7. 2026M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
  8. 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text