# End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding

arXiv: https://arxiv.org/abs/2405.13527 · HTML: https://mlsanity.com/paper/2405.13527/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 22 references that are in the map.
- First posted on arXiv 22 May 2024.

## Builds on (most cited first, 7 of 7)

- [A Survey on Multi-Task Learning](https://mlsanity.com/paper/1707.08114/) (2017)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Professor Forcing: A New Algorithm for Training Recurrent Networks](https://mlsanity.com/paper/1610.09038/) (2016)
- [A holistic approach to polyphonic music transcription with neural networks](https://mlsanity.com/paper/1910.12086/) (2019)
- [Non-Local Musical Statistics as Guides for Audio-to-Score Piano Transcription](https://mlsanity.com/paper/2008.12710/) (2020)
- [Evaluating Non-aligned Musical Score Transcriptions with MV2H](https://mlsanity.com/paper/1906.00566/) (2019)

## Built on by (most cited first, 5 of 5)

- [Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcription](https://mlsanity.com/paper/2509.23878/) (2025)
- [Joint Estimation of Piano Dynamics and Metrical Structure with a Multi-task Multi-Scale Network](https://mlsanity.com/paper/2510.18190/) (2025)
- [Rubato: Transcribing Piano Music with Timestamps](https://mlsanity.com/paper/2605.24291/) (2026)
- [Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrapping](https://mlsanity.com/paper/2608.03050/) (2026)
- [TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data](https://mlsanity.com/paper/2609.00640/) (2026)

## Abstract (arXiv)

Piano audio-to-score transcription (A2S) is an important yet underexplored task with extensive applications for music composition, practice, and analysis. However, existing end-to-end piano A2S systems faced difficulties in retrieving bar-level information such as key and time signatures, and have been trained and evaluated with only synthetic data. To address these limitations, we propose a sequence-to-sequence (Seq2Seq) model with a hierarchical decoder that aligns with the hierarchical structure of musical scores, enabling the transcription of score information at both the bar and note levels by multi-task learning. To bridge the gap between synthetic data and recordings of human performance, we propose a two-stage training scheme, which involves pre-training the model using an expressive performance rendering (EPR) system on synthetic audio, followed by fine-tuning the model using recordings of human performance. To preserve the voicing structure for score reconstruction, we propose a pre-processing method for **Kern scores in scenarios with an unconstrained number of voices. Experimental results support the effectiveness of our proposed approaches, in terms of both transcription performance on synthetic audio data in comparison to the current state-of-the-art, and the first experiment on human recordings.