# Unaligned Supervision For Automatic Music Transcription in The Wild

arXiv: https://arxiv.org/abs/2204.13668 · HTML: https://mlsanity.com/paper/2204.13668/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 19 references that are in the map.
- First posted on arXiv 28 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning](https://mlsanity.com/paper/1704.03976/) (2017)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [MT3: Multi-Task Multitrack Music Transcription](https://mlsanity.com/paper/2111.03017/) (2021)
- [High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times](https://mlsanity.com/paper/2010.01815/) (2020)
- [Learning Features of Music from Scratch](https://mlsanity.com/paper/1611.09827/) (2016)
- [Sequence-to-Sequence Piano Transcription with Transformers](https://mlsanity.com/paper/2107.09142/) (2021)
- [ReconVAT: A Semi-Supervised Automatic Music Transcription Framework for Low-Resource Real-World Data](https://mlsanity.com/paper/2107.04954/) (2021)
- [Simultaneous Separation and Transcription of Mixtures with Multiple Polyphonic and Percussive Instruments](https://mlsanity.com/paper/1910.12621/) (2019)
- [Invariances and Data Augmentation for Supervised Music Transcription](https://mlsanity.com/paper/1711.04845/) (2017)

## Built on by (most cited first, 25 of 25)

- [LLark: A Multimodal Instruction-Following Language Model for Music](https://mlsanity.com/paper/2310.07160/) (2023)
- [A Data-Driven Analysis of Robust Automatic Piano Transcription](https://mlsanity.com/paper/2402.01424/) (2024)
- [High Resolution Guitar Transcription via Domain Adaptation](https://mlsanity.com/paper/2402.15258/) (2024)
- [DiffRoll: Diffusion-based Generative Music Transcription with Unsupervised Pretraining Capability](https://mlsanity.com/paper/2210.05148/) (2022)
- [YourMT3+: Multi-instrument Music Transcription with Enhanced Transformer Architectures and Cross-dataset Stem Augmentation](https://mlsanity.com/paper/2407.04822/) (2024)
- [Unified Cross-modal Translation of Score Images, Symbolic Music, and Performance Audio](https://mlsanity.com/paper/2505.12863/) (2025)
- [Performance Conditioning for Diffusion-Based Multi-Instrument Music Synthesis](https://mlsanity.com/paper/2309.12283/) (2023)
- [Toward Fully Self-Supervised Multi-Pitch Estimation](https://mlsanity.com/paper/2402.15569/) (2024)
- [PianoMotion10M: Dataset and Benchmark for Hand Motion Generation in Piano Performance](https://mlsanity.com/paper/2406.09326/) (2024)
- [Annotation-free Automatic Music Transcription with Scalable Synthetic Data and Adversarial Domain Confusion](https://mlsanity.com/paper/2312.10402/) (2023)
- [Just Label the Repeats for In-The-Wild Audio-to-Score Alignment](https://mlsanity.com/paper/2411.07428/) (2024)
- [Count The Notes: Histogram-Based Supervision for Automatic Music Transcription](https://mlsanity.com/paper/2511.14250/) (2025)
- [MuScriptor: An Open Model for Multi-Instrument Music Transcription](https://mlsanity.com/paper/2607.08168/) (2026)
- [Analyzing and reducing the synthetic-to-real transfer gap in Music Information Retrieval: the task of automatic drum transcription](https://mlsanity.com/paper/2407.19823/) (2024)
- [Evaluating Disentangled Representations for Controllable Music Generation](https://mlsanity.com/paper/2602.10058/) (2026)
- [Music Transcription with (Almost) No Supervision](https://mlsanity.com/paper/2605.24193/) (2026)
- [Transfer of knowledge among instruments in automatic music transcription](https://mlsanity.com/paper/2305.00426/) (2023)
- [Fine-Tuning MIDI-to-Audio Alignment using a Neural Network on Piano Roll and CQT Representations](https://mlsanity.com/paper/2506.22237/) (2025)
- [Investigating an Overfitting and Degeneration Phenomenon in Self-Supervised Multi-Pitch Estimation](https://mlsanity.com/paper/2506.23371/) (2025)
- [Velocity Prediction in Automatic Guitar Transcription](https://mlsanity.com/paper/2606.24912/) (2026)
- [Decomposer: Learning to Decompile Symbolic Music to Programs](https://mlsanity.com/paper/2607.01849/) (2026)
- [Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion](https://mlsanity.com/paper/2607.13278/) (2026)
- [Generalized Audio-Driven Synthesis of Precise Drummer Motion](https://mlsanity.com/paper/2608.19055/) (2026)
- [TART: A Modular Tool for Technique-Aware Audio-to-Tablature Guitar Transcription](https://mlsanity.com/paper/2609.11904/) (2026)
- [Synthesis and editing of multi-instrument audio mixtures using scalar-quantised latents with MIDI Span conditioning](https://mlsanity.com/paper/2609.25546/) (2026)

## Abstract (arXiv)

Multi-instrument Automatic Music Transcription (AMT), or the decoding of a musical recording into semantic musical content, is one of the holy grails of Music Information Retrieval. Current AMT approaches are restricted to piano and (some) guitar recordings, due to difficult data collection. In order to overcome data collection barriers, previous AMT approaches attempt to employ musical scores in the form of a digitized version of the same song or piece. The scores are typically aligned using audio features and strenuous human intervention to generate training labels. We introduce NoteEM, a method for simultaneously training a transcriber and aligning the scores to their corresponding performances, in a fully-automated process. Using this unaligned supervision scheme, complemented by pseudo-labels and pitch-shift augmentation, our method enables training on in-the-wild recordings with unprecedented accuracy and instrumental variety. Using only synthetic data and unaligned supervision, we report SOTA note-level accuracy of the MAPS dataset, and large favorable margins on cross-dataset evaluations. We also demonstrate robustness and ease of use; we report comparable results when training on a small, easily obtainable, self-collected dataset, and we propose alternative labeling to the MusicNet dataset, which we show to be more accurate. Our project page is available at https://benadar293.github.io