# Teaching Machines to Code: Neural Markup Generation with Visual Attention

arXiv: https://arxiv.org/abs/1802.05415 · HTML: https://mlsanity.com/paper/1802.05415/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2018.

## Built on by (most cited first, 5 of 5)

- [Nougat: Neural Optical Understanding for Academic Documents](https://mlsanity.com/paper/2308.13418/) (2023)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [MathNet: A Data-Centric Approach for Printed Mathematical Expression Recognition](https://mlsanity.com/paper/2404.13667/) (2024)
- [From Robots to Books: An Introduction to Smart Applications of AI in Education (AIEd)](https://mlsanity.com/paper/2301.10026/) (2023)
- [DREAM: Document Reconstruction via End-to-end Autoregressive Model](https://mlsanity.com/paper/2507.05805/) (2025)

## Abstract (arXiv)

We present a neural transducer model with visual attention that learns to generate LaTeX markup of a real-world math formula given its image. Applying sequence modeling and transduction techniques that have been very successful across modalities such as natural language, image, handwriting, speech and audio; we construct an image-to-markup model that learns to produce syntactically and semantically correct LaTeX markup code over 150 words long and achieves a BLEU score of 89%; improving upon the previous state-of-art for the Im2Latex problem. We also demonstrate with heat-map visualization how attention helps in interpreting the model and can pinpoint (detect and localize) symbols on the image accurately despite having been trained without any bounding box data.