mlsanityData to 14 Oct 2026

UniMERNet: A Universal Network for Real-World Mathematical Expression Recognition

arXiv 2404.15254 ↗First posted 23 Apr 2024cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

The paper introduces the UniMER dataset, marking the first study on Mathematical Expression Recognition (MER) targeting complex real-world scenarios. The UniMER dataset includes a large-scale training set, UniMER-1M, which offers unprecedented scale and diversity with one million training instances to train high-quality, robust models. Additionally, UniMER features a meticulously designed, diverse test set, UniMER-Test, which covers a variety of formula distributions found in real-world scenarios, providing a more comprehensive and fair evaluation. To better utilize the UniMER dataset, the paper proposes a Universal Mathematical Expression Recognition Network (UniMERNet), tailored to the characteristics of formula recognition. UniMERNet consists of a carefully designed encoder that incorporates detail-aware and local context features, and an optimized decoder for accelerated performance. Extensive experiments conducted using the UniMER-1M dataset and UniMERNet demonstrate that training on the large-scale UniMER-1M dataset can produce a more generalizable formula recognition model, significantly outperforming all previous datasets. Furthermore, the introduction of UniMERNet enhances the model's performance in formula recognition, achieving higher accuracy and speeds. All data, models, and code are available at https://github.com/opendatalab/UniMERNet.

Key facts. Cited by 42 papers in the map (data through 14 Oct 2026). Builds on 22 of its 42 references that are in the map. First posted on arXiv 23 Apr 2024.

Builds on

22 of 42 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2021OCR-free Document Understanding Transformercited by 407
  8. 2021Conditional Positional Encodings for Vision Transformerscited by 406
  9. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  10. 2021CMT: Convolutional Neural Networks Meet Vision Transformerscited by 310
  11. 2023Nougat: Neural Optical Understanding for Academic Documentscited by 245
  12. 2023Vary: Scaling up the Vision Vocabulary for Large Vision-Language Modelscited by 84
10 more
  1. 2023VIGC: Visual Instruction Generation and Correctioncited by 80
  2. 2016Image-to-Markup Generation with Coarse-to-Fine Attentioncited by 79
  3. 2022Syntax-Aware Network for Handwritten Mathematical Expression Recognitioncited by 68
  4. 2018Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognitioncited by 36
  5. 2022When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognitioncited by 35
  6. 2021Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformercited by 34
  7. 2022CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognitioncited by 24
  8. 2020Improving Attention-Based Handwritten Mathematical Expression Recognition with Scale Augmentation and Drop Attentioncited by 22
  9. 2021Handwritten Mathematical Expression Recognition via Attention Aggregation based Bi-directional Mutual Learningcited by 20
  10. 2019Pattern Generation Strategies for Improving Recognition of Handwritten Mathematical Expressionscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 14; 2026: 19.

42 papers.

Sort:

By area:

  1. 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
  2. 2025PaddleOCR-VL: Boosting Multilingual Document Parsing via a 0.9B Ultra-Compact Vision-Language Modelcited by 89
  3. 2025MonkeyOCR: Document Parsing with a Structure-Recognition-Relation Triplet Paradigmcited by 87
  4. 2025Dolphin: Document Image Parsing via Heterogeneous Anchor Promptingcited by 79
  5. 2024Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extractioncited by 44
  6. 2024Image Over Text: Transforming Formula Recognition Evaluation with Character Detection Matchingcited by 41
  7. 2024DocGenome: An Open Large-scale Scientific Document Benchmark for Training and Testing Multi-modal Large Language Modelscited by 36
  8. 2024TextHawk2: A Large Vision-Language Model Excels in Bilingual OCR and Grounding with 16x Fewer Tokenscited by 32
  9. 2026MinerU2.5-Pro: Pushing the Limits of Data-Centric Document Parsing at Scalecited by 27
  10. 2024OCR Hinders RAG: Evaluating the Cascading Impact of OCR on Retrieval-Augmented Generationcited by 26
  11. 2024GeoX: Geometric Problem Solving Through Unified Formalized Vision-Language Pre-trainingcited by 23
  12. 2025Marten: Visual Question Answering with Mask Generation for Multi-modal Document Understandingcited by 20
30 more
  1. 2024PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformercited by 18
  2. 2025MonkeyOCR v1.5 Technical Report: Unlocking Robust Document Parsing for Complex Patternscited by 13
  3. 2025UniRec-0.1B: Unified Text and Formula Recognition with 0.1B Parameterscited by 13
  4. 2025DianJin-OCR-R1: Enhancing OCR Capabilities via a Reasoning-and-Tool Interleaved Vision-Language Modelcited by 10
  5. 2026MinerU-Diffusion: Rethinking Document OCR as Inverse Rendering via Diffusion Decodingcited by 8
  6. 2025PP-FormulaNet: Bridging Accuracy and Efficiency in Advanced Formula Recognitioncited by 7
  7. 2026Towards Real-World Document Parsing via Realistic Scene Synthesis and Document-Aware Trainingcited by 7
  8. 2025DOCR-Inspector: Fine-Grained and Automated Evaluation of Document Parsing with VLMcited by 6
  9. 2026Beyond NL2Code: A Structured Survey of Multimodal Code Intelligencecited by 6
  10. 2026MDPBench: A Benchmark for Multilingual Document Parsing in Real-World Scenarioscited by 5
  11. 2026Infinity-Parser2 Technical Reportcited by 4
  12. 2026MonkeyOCRv2: A Visual-Text Foundation Model for Document AIcited by 4
  13. 2025Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognitioncited by 3
  14. 2025DocTron-Formula: Generalized Formula Recognition in Complex and Structured Scenarioscited by 3
  15. 2025MindVL: Towards Efficient and Effective Training of Multimodal Large Language Models on Ascend NPUscited by 3
  16. 2026Boosting Document Parsing Efficiency and Performance with Coarse-to-Fine Visual Processingcited by 3
  17. 2025HyperVL: An Efficient and Dynamic Multimodal Large Language Model for Edge Devicescited by 2
  18. 2026VisTIRA: Closing the Image-Text Modality Gap in Visual Math Reasoning via Structured Tool Integrationcited by 2
  19. 2026Efficient Document Parsing via Parallel Token Predictioncited by 2
  20. 2026SEA-Vision: A Multilingual Benchmark for Comprehensive Document and Scene Text Understanding in Southeast Asiacited by 2
  21. 2026ChartArena: Benchmarking Chart Parsing across Languages, Scenarios, and Formatscited by 2
  22. 2024Enhancing Complex Formula Recognition with Hierarchical Detail-Focused Networkcited by 1
  23. 2026Q-Mask: Query-driven Causal Masks for Text Anchoring in OCR-Oriented Vision-Language Modelscited by 1
  24. 2025Towards Scalable Training for Handwritten Mathematical Expression Recognitioncited by 0
  25. 2026MeDocVL: A Visual Language Model for Medical Document Understanding and Parsingcited by 0
  26. 2026Texo: Formula Recognition within 20M Parameterscited by 0
  27. 2026PositionOCR: Augmenting Positional Awareness in Multi-Modal Models via Hybrid Specialist Integrationcited by 0
  28. 2026PA-CDM: Position-Aware Character Detection Matching for Evaluating Handwritten Mathematical Expression Recognitioncited by 0
  29. 2026Diffusion Drafts, AR Verifies: Accelerating Document OCR with Self-Speculative Decodingcited by 0
  30. 2026Xiaomi-OCR-0 Technical Reportcited by 0