DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarios
Not on the map. Why?
Abstract (arXiv)
Optical Character Recognition (OCR) for mathematical formula is essential for the intelligent analysis of scientific literature. However, both task-specific and general vision-language models often struggle to handle the structural diversity, complexity, and real-world variability inherent in mathematical content. In this work, we present DocTron-Formula, a unified framework built upon general vision-language models, thereby eliminating the need for specialized architectures. Furthermore, we introduce CSFormula, a large-scale and challenging dataset that encompasses multidisciplinary and structurally complex formulas at the line, paragraph, and page levels. Through straightforward supervised fine-tuning, our approach achieves state-of-the-art performance across a variety of styles, scientific domains, and complex layouts. Experimental results demonstrate that our method not only surpasses specialized models in terms of accuracy and robustness, but also establishes a new paradigm for the automated understanding of complex scientific documents.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 26 of its 40 references that are in the map. First posted on arXiv 1 Aug 2025.
Builds on
26 of 40 references.
References · 26
- 2017Attention Is All You Needcited by 60,361
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
- 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
- 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
- 2021OCR-free Document Understanding Transformercited by 407
- 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 274
- 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
- 2023Honeybee: Locality-enhanced Projector for Multimodal LLMcited by 197
14 more
- 2023SVIT: Scaling up Visual Instruction Tuningcited by 162
- 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
- 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
- 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
- 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
- 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
- 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 36
- 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
- 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
- 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
- 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
- 2019Pattern Generation Strategies for Improving Recognition of Handwritten Mathematical Expressionscited by 13
- 2025PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognitioncited by 7
- 2024Enhancing Complex Formula Recognition with Hierarchical Detail-Focused Networkcited by 1
Built on by
Citing papers per year: 2026: 3.
3 papers.
Similar, not cited
Close in wording, no citation either way
- 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameters
- 2024STEM-POM: Evaluating Language Models Math-Symbol Reasoning in Document Parsing
- 2026Dr. DocBench: A Comprehensive Benchmark for Expert-Level and Difficult Document Parsing
- 2024ChatSR: Multimodal Large Language Models for Scientific Formula Discovery
- 2024MathWriting: A Dataset For Handwritten Mathematical Expression Recognition
- 2026Beyond Text and Tables: Vision-Language Model Integration in ComProScanner for Extracting Materials Data from Scientific Figures with High Accuracy
- 2026M3Kang: Evaluating Multilingual Multimodal Mathematical Reasoning in Vision-Language Models
- 2023OpenWebMath: An Open Dataset of High-Quality Mathematical Web Text