# Sheet Music Transformer: End-To-End Optical Music Recognition Beyond Monophonic Transcription

arXiv: https://arxiv.org/abs/2402.07596 · HTML: https://mlsanity.com/paper/2402.07596/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 38 references that are in the map.
- First posted on arXiv 12 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [An End-to-End Trainable Neural Network for Image-based Sequence Recognition and Its Application to Scene Text Recognition](https://mlsanity.com/paper/1507.05717/) (2015)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [DAN: a Segmentation-free Document Attention Network for Handwritten Document Recognition](https://mlsanity.com/paper/2203.12273/) (2022)
- [Understanding Optical Music Recognition](https://mlsanity.com/paper/1908.03608/) (2019)
- [An Efficient End-to-End Neural Model for Handwritten Text Recognition](https://mlsanity.com/paper/1807.07965/) (2018)
- [Full Page Handwriting Recognition via Image to Sequence Extraction](https://mlsanity.com/paper/2103.06450/) (2021)
- [A holistic approach to polyphonic music transcription with neural networks](https://mlsanity.com/paper/1910.12086/) (2019)
- [An Empirical Evaluation of End-to-End Polyphonic Optical Music Recognition](https://mlsanity.com/paper/2108.01769/) (2021)
- [MSdocTr-Lite: A Lite Transformer for Full Page Multi-script Handwriting Recognition](https://mlsanity.com/paper/2303.13931/) (2023)
- [A Unified Representation Framework for the Evaluation of Optical Music Recognition Systems](https://mlsanity.com/paper/2312.12908/) (2023)

## Built on by (most cited first, 20 of 20)

- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [Sheet Music Benchmark: Standardized Optical Music Recognition Evaluation](https://mlsanity.com/paper/2506.10488/) (2025)
- [Unified Cross-modal Translation of Score Images, Symbolic Music, and Performance Audio](https://mlsanity.com/paper/2505.12863/) (2025)
- [Towards an AI Musician: Synthesizing Sheet Music Problems for Musical Reasoning](https://mlsanity.com/paper/2509.04059/) (2025)
- [Knowledge Discovery in Optical Music Recognition: Enhancing Information Retrieval with Instance Segmentation](https://mlsanity.com/paper/2408.15002/) (2024)
- [MusiXQA: Advancing Visual Music Understanding in Multimodal Large Language Models](https://mlsanity.com/paper/2506.23009/) (2025)
- [Synthesising Handwritten Music with GANs: A Comprehensive Evaluation of CycleWGAN, ProGAN, and DCGAN](https://mlsanity.com/paper/2411.16405/) (2024)
- [LEGATO: Large-scale End-to-end Generalizable Approach to Typeset OMR](https://mlsanity.com/paper/2506.19065/) (2025)
- [Teaching LLMs Music Theory with In-Context Learning and Chain-of-Thought Prompting: Pedagogical Strategies for Machines](https://mlsanity.com/paper/2503.22853/) (2025)
- [Optical Music Recognition of Jazz Lead Sheets](https://mlsanity.com/paper/2509.05329/) (2025)
- [Low-Data Classification of Historical Music Manuscripts: A Few-Shot Learning Approach](https://mlsanity.com/paper/2411.16408/) (2024)
- [Transcoda: End-to-End Zero-Shot Optical Music Recognition via Data-Centric Synthetic Training](https://mlsanity.com/paper/2605.10835/) (2026)
- [Direct content-based retrieval from music scores images](https://mlsanity.com/paper/2605.22255/) (2026)
- [Optical Music Recognition for Real-World Manuscripts with Synthetic Data](https://mlsanity.com/paper/2606.09479/) (2026)
- [A document is worth a structured record: Principled inductive bias design for document recognition](https://mlsanity.com/paper/2507.08458/) (2025)
- [A High-Accuracy Optical Music Recognition Method Based on Bottleneck Residual Convolutions](https://mlsanity.com/paper/2604.16446/) (2026)
- [Towards a foundational model for recognising diastematic Gregorian notation](https://mlsanity.com/paper/2606.31454/) (2026)
- [LEGATO 2: Toward Multimodal Sheet Music Recognition and Understanding](https://mlsanity.com/paper/2607.05769/) (2026)
- [A Dataset and Benchmark for Optical Music Recognition of String Quartet Scores](https://mlsanity.com/paper/2608.10978/) (2026)
- [Exploring In-Context Learning for Handwritten Text Recognition](https://mlsanity.com/paper/2609.37195/) (2026)

## Abstract (arXiv)

State-of-the-art end-to-end Optical Music Recognition (OMR) has, to date, primarily been carried out using monophonic transcription techniques to handle complex score layouts, such as polyphony, often by resorting to simplifications or specific adaptations. Despite their efficacy, these approaches imply challenges related to scalability and limitations. This paper presents the Sheet Music Transformer, the first end-to-end OMR model designed to transcribe complex musical scores without relying solely on monophonic strategies. Our model employs a Transformer-based image-to-sequence framework that predicts score transcriptions in a standard digital music encoding format from input images. Our model has been tested on two polyphonic music datasets and has proven capable of handling these intricate music structures effectively. The experimental outcomes not only indicate the competence of the model, but also show that it is better than the state-of-the-art methods, thus contributing to advancements in end-to-end OMR transcription.