# MMHMER:Multi-viewer and Multi-task for Handwritten Mathematical Expression Recognition

arXiv: https://arxiv.org/abs/2502.05557 · HTML: https://mlsanity.com/paper/2502.05557/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 31 references that are in the map.
- First posted on arXiv 8 Feb 2025.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Learned Image Compression with Mixed Transformer-CNN Architectures](https://mlsanity.com/paper/2303.14978/) (2023)
- [Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/1801.03530/) (2018)
- [When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2207.11463/) (2022)
- [Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer](https://mlsanity.com/paper/2105.02412/) (2021)
- [A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation](https://mlsanity.com/paper/2307.12574/) (2023)
- [CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2207.04410/) (2022)
- [PosFormer: Recognizing Complex Handwritten Mathematical Expression with Position Forest Transformer](https://mlsanity.com/paper/2407.07764/) (2024)
- [A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/1712.03991/) (2017)
- [Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2308.10493/) (2023)
- [NAMER: Non-Autoregressive Modeling for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2407.11380/) (2024)
- [CNN-transformer mixed model for object detection](https://mlsanity.com/paper/2212.06714/) (2022)

## Built on by (most cited first, 1 of 1)

- [Uni-MuMER: Unified Multi-Task Fine-Tuning of Vision-Language Model for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2505.23566/) (2025)

## Abstract (arXiv)

Handwritten Mathematical Expression Recognition (HMER) methods have made remarkable progress, with most existing HMER approaches based on either a hybrid CNN/RNN-based with GRU architecture or Transformer architectures. Each of these has its strengths and weaknesses. Leveraging different model structures as viewers and effectively integrating their diverse capabilities presents an intriguing avenue for exploration. This involves addressing two key challenges: 1) How to fuse these two methods effectively, and 2) How to achieve higher performance under an appropriate level of complexity. This paper proposes an efficient CNN-Transformer multi-viewer, multi-task approach to enhance the model's recognition performance. Our MMHMER model achieves 63.96%, 62.51%, and 65.46% ExpRate on CROHME14, CROHME16, and CROHME19, outperforming Posformer with an absolute gain of 1.28%, 1.48%, and 0.58%. The main contribution of our approach is that we propose a new multi-view, multi-task framework that can effectively integrate the strengths of CNN and Transformer. By leveraging the feature extraction capabilities of CNN and the sequence modeling capabilities of Transformer, our model can better handle the complexity of handwritten mathematical expressions.