Towards Scalable Training for Handwritten Mathematical Expression Recognition

Large foundation models have achieved significant performance gains through scalable training on massive datasets. However, the field of Handwritten Mathematical Expression Recognition (HMER) has been impeded by the scarcity of data, primarily due to the arduous and costly process of manual annotation. To bridge this gap, we propose a novel method integrating limited handwritten formulas with large-scale LaTeX-rendered formulas by developing a scalable data engine to generate complex and consistent LaTeX sequences. With this engine, we built the largest formula dataset to date, termed textttTex80M, comprising over 80 million high-quality training instances. Then we propose textttTexTeller, the first HMER model trained at scale, by mix-training textttTex80M with a relatively small HME dataset. The expansive training dataset and our refined pipeline have equipped textttTexTeller with state-of-the-art (SOTA) performance across nearly all benchmarks. To advance the field, we will openly release our complete model, entire dataset, and full codebase, enabling further research building upon our contributions.