# TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data

arXiv: https://arxiv.org/abs/2609.00640 · HTML: https://mlsanity.com/paper/2609.00640/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 13 of its 32 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases](https://mlsanity.com/paper/2103.10697/) (2021)
- [NotaGen: Advancing Musicality in Symbolic Music Generation with Large Language Model Training Paradigms](https://mlsanity.com/paper/2502.18008/) (2025)
- [TunesFormer: Forming Irish Tunes with Control Codes by Bar Patching](https://mlsanity.com/paper/2301.02884/) (2023)
- [MelodyT5: A Unified Score-to-Score Transformer for Symbolic Music Processing](https://mlsanity.com/paper/2407.02277/) (2024)
- [A holistic approach to polyphonic music transcription with neural networks](https://mlsanity.com/paper/1910.12086/) (2019)
- [Non-Local Musical Statistics as Guides for Audio-to-Score Piano Transcription](https://mlsanity.com/paper/2008.12710/) (2020)
- [Online Symbolic Music Alignment with Offline Reinforcement Learning](https://mlsanity.com/paper/2401.00466/) (2023)
- [End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding](https://mlsanity.com/paper/2405.13527/) (2024)
- [SoulX-Singer: Towards High-Quality Zero-Shot Singing Voice Synthesis](https://mlsanity.com/paper/2602.07803/) (2026)

## Abstract (arXiv)

Generalizable Audio-to-Score (A2S) transcription is fundamentally constrained by the severe scarcity of high-quality, real-world paired data. Relying solely on existing human-annotated datasets often restricts the generalization of A2S models, limiting their efficacy primarily to single-instrumentation domains. To break this dependency on scarce real-world data, we introduce TUTTI (Transformer for Unified audio-To-score Transcription trained on Synthetic multi-Instrumentation Data), a pre-training paradigm driven by a purely synthetic, large-scale dataset. Rather than using human-composed scores, we leverage a symbolic music generation model to generate a massive, highly scalable multi-instrumentation corpus and create audio-score pairs with expressive acoustic characteristics. Capitalizing on the generated data, we employ a standard Transformer encoder-decoder architecture. We empirically demonstrate that pre-training a unified attention-based model on generated, multi-instrumentation data yields a consistently stronger foundational representation than single-instrumentation training. When fine-tuned with downstream real-world datasets, TUTTI outperforms previous approaches, establishing new overall state-of-the-art results across various A2S baselines. Notably, TUTTI shows remarkable cross-instrument transferability, effectively adapting to unseen instruments with highly competitive performance. The source code and the TuttiCorpus dataset will be made publicly available at https://github.com/a-musiclover/TUTTI.