# Bidirectional Trained Tree-Structured Decoder for Handwritten Mathematical Expression Recognition

arXiv: https://arxiv.org/abs/2401.00435 · HTML: https://mlsanity.com/paper/2401.00435/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 50 references that are in the map.
- First posted on arXiv 31 Dec 2023.

## Builds on (most cited first, 20 of 20)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Bidirectional LSTM-CRF Models for Sequence Tagging](https://mlsanity.com/paper/1508.01991/) (2015)
- [Towards Accurate Scene Text Recognition with Semantic Reasoning Networks](https://mlsanity.com/paper/2003.12294/) (2020)
- [SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognition](https://mlsanity.com/paper/2005.10977/) (2020)
- [NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition](https://mlsanity.com/paper/1806.00926/) (2018)
- [Syntax-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2203.01601/) (2022)
- [Split, embed and merge: An accurate table structure recognizer](https://mlsanity.com/paper/2107.05214/) (2021)
- [ABINet++: Autonomous, Bidirectional and Iterative Language Modeling for Scene Text Spotting](https://mlsanity.com/paper/2211.10578/) (2022)
- [Multi-Scale Attention with Dense Encoder for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/1801.03530/) (2018)
- [When Counting Meets HMER: Counting-Aware Network for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2207.11463/) (2022)
- [Handwritten Mathematical Expression Recognition with Bidirectionally Trained Transformer](https://mlsanity.com/paper/2105.02412/) (2021)
- [CoMER: Modeling Coverage for Transformer-based Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2207.04410/) (2022)
- [Asynchronous Bidirectional Decoding for Neural Machine Translation](https://mlsanity.com/paper/1801.05122/) (2018)
- [Handwritten Mathematical Expression Recognition via Attention Aggregation based Bi-directional Mutual Learning](https://mlsanity.com/paper/2112.03603/) (2021)
- [Synchronous Bidirectional Inference for Neural Sequence Generation](https://mlsanity.com/paper/1902.08955/) (2019)
- [Multimodal Tree Decoder for Table of Contents Extraction in Document Images](https://mlsanity.com/paper/2212.02896/) (2022)
- [Semantic Graph Representation Learning for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2308.10493/) (2023)
- [Stroke Constrained Attention Network for Online Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2002.08670/) (2020)
- [Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction](https://mlsanity.com/paper/2307.16253/) (2023)

## Built on by (most cited first, 1 of 1)

- [NAMER: Non-Autoregressive Modeling for Handwritten Mathematical Expression Recognition](https://mlsanity.com/paper/2407.11380/) (2024)

## Abstract (arXiv)

The Handwritten Mathematical Expression Recognition (HMER) task is a critical branch in the field of OCR. Recent studies have demonstrated that incorporating bidirectional context information significantly improves the performance of HMER models. However, existing methods fail to effectively utilize bidirectional context information during the inference stage. Furthermore, current bidirectional training methods are primarily designed for string decoders and cannot adequately generalize to tree decoders, which offer superior generalization capabilities and structural analysis capacity. In order to overcome these limitations, we propose the Mirror-Flipped Symbol Layout Tree (MF-SLT) and Bidirectional Asynchronous Training (BAT) structure. Our method extends the bidirectional training strategy to the tree decoder, allowing for more effective training by leveraging bidirectional information. Additionally, we analyze the impact of the visual and linguistic perception of the HMER model separately and introduce the Shared Language Modeling (SLM) mechanism. Through the SLM, we enhance the model's robustness and generalization when dealing with visual ambiguity, particularly in scenarios with abundant training data. Our approach has been validated through extensive experiments, demonstrating its ability to achieve new state-of-the-art results on the CROHME 2014, 2016, and 2019 datasets, as well as the HME100K dataset. The code used in our experiments will be publicly available.