mlsanityData to 14 Oct 2026

Image-to-Markup Generation with Coarse-to-Fine Attention

arXiv 1609.04938 ↗First posted 16 Sep 2016cs.CV cs.CL cs.LGcited by 79

Not on the map. Why?

Abstract (arXiv)

We present a neural encoder-decoder model to convert images into presentational markup based on a scalable coarse-to-fine attention mechanism. Our method is evaluated in the context of image-to-LaTeX generation, and we introduce a new dataset of real-world rendered mathematical expressions paired with LaTeX markup. We show that unlike neural OCR techniques using CTC-based models, attention-based approaches can tackle this non-standard OCR task. Our approach outperforms classical mathematical OCR systems by a large margin on in-domain rendered data, and, with pretraining, also performs well on out-of-domain handwritten data. To reduce the inference complexity associated with the attention-based approaches, we introduce a new coarse-to-fine attention layer that selects a support region before applying attention.

Key facts. Cited by 79 papers in the map (data through 14 Oct 2026). First posted on arXiv 16 Sep 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 10; 2022: 8; 2023: 10; 2024: 18; 2025: 16; 2026: 17.

79 papers.

Sort:

By area:

  1. 2022Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understandingcited by 366
  2. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  3. 2024OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotationscited by 150
  4. 2023KOSMOS-2.5: A Multimodal Literate Modelcited by 96
  5. 2024OmniParser: A Unified Framework for Text Spotting, Key Information Extraction and Table Recognitioncited by 74
  6. 2022Syntax-Aware Network for Handwritten Mathematical Expression Recognitioncited by 68
  7. 2025Ming-Omni: A Unified Multimodal Model for Perception and Generationcited by 66
  8. 2021LayoutParser: A Unified Toolkit for Deep Learning Based Document Image Analysiscited by 58
  9. 2025SmolDocling: An ultra-compact vision-language model for end-to-end multi-modal document conversioncited by 53
  10. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  11. 2024UniMERNet: A Universal Network for Real-World Mathematical Expression Recognitioncited by 42
  12. 2023Improving Table Structure Recognition with Visual-Alignment Sequential Coordinate Modelingcited by 41
38 more
  1. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  2. 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
  3. 2024FlowLearn: Evaluating Large Vision-Language Models on Flowchart Understandingcited by 27
  4. 2021Full Page Handwriting Recognition via Image to Sequence Extractioncited by 24
  5. 2025OmniParser V2: Structured-Points-of-Thought for Unified Visual Text Parsing and Its Generality to Multimodal Large Language Modelscited by 23
  6. 2024MathWriting: A Dataset For Handwritten Mathematical Expression Recognitioncited by 21
  7. 2021Handwritten Mathematical Expression Recognition via Attention Aggregation based Bi-directional Mutual Learningcited by 20
  8. 2021Current Status and Performance Analysis of Table Recognition in Document Images with Deep Neural Networkscited by 18
  9. 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
  10. 2021Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documentscited by 17
  11. 2021TabLeX: A Benchmark Dataset for Structure and Content Information Extraction from Scientific Tablescited by 15
  12. 2024READoc: A Unified Benchmark for Realistic Document Structured Extractioncited by 15
  13. 2023An End-to-End Multi-Task Learning Model for Image-based Table Recognitioncited by 14
  14. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  15. 2023Rethinking Image-based Table Recognition Using Weakly Supervised Methodscited by 10
  16. 2021ICDAR 2021 Competition on Scientific Table Image Recognition to LaTeXcited by 9
  17. 2024LATTE: Improving Latex Recognition for Tables and Formulae with Iterative Refinementcited by 8
  18. 2022Tables to LaTeX: structure and content extraction from scientific tablescited by 7
  19. 2023Neural Machine Translation for Mathematical Formulaecited by 7
  20. 2025PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognitioncited by 7
  21. 2025TeXpert: A Multi-Level Benchmark for Evaluating LaTeX Code Generation by LLMscited by 7
  22. 2022SemEval 2022 Task 12: Symlink- Linking Mathematical Symbols to their Descriptionscited by 6
  23. 2023Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognitioncited by 4
  24. 2024MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognitioncited by 4
  25. 2024Leveraging Label Semantics and Meta-Label Refinement for Multi-Label Question Classificationcited by 4
  26. 2025VEHME: A Vision-Language Model For Evaluating Handwritten Mathematics Expressionscited by 4
  27. 2025Benchmarking Document Parsers on Mathematical Formula Extraction from PDFscited by 4
  28. 2026When VLMs 'Fix' Students: Identifying and Penalizing Over-Correction in the Evaluation of Multi-line Handwritten Math OCRcited by 4
  29. 2026Infinity-Parser2 Technical Reportcited by 4
  30. 2021Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attentioncited by 3
  31. 2023SCOB: Universal Text Understanding via Character-wise Supervised Contrastive Learning with Online Text Rendering for Bridging Domain Gapcited by 3
  32. 2024NAMER: Non-Autoregressive Modeling for Handwritten Mathematical Expression Recognitioncited by 3
  33. 2025Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognitioncited by 3
  34. 2025DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarioscited by 3
  35. 2021Deep Structured Feature Networks for Table Detection and Tabular Data Extraction from Scanned Financial Document Imagescited by 2
  36. 2022SemSup: Semantic Supervision for Simple and Scalable Zero-shot Generalizationcited by 2
  37. 2022Symlink: A New Dataset for Scientific Symbol-Description Linkingcited by 2
  38. 2025DREAM: Document Reconstruction via End-to-end Autoregressive Modelcited by 2

50 most cited of 79 All 79 (JSON)