# Fine-Tuning MIDI-to-Audio Alignment using a Neural Network on Piano Roll and CQT Representations

arXiv: https://arxiv.org/abs/2506.22237 · HTML: https://mlsanity.com/paper/2506.22237/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 8 of its 27 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Unaligned Supervision For Automatic Music Transcription in The Wild](https://mlsanity.com/paper/2204.13668/) (2022)
- [High Resolution Guitar Transcription via Domain Adaptation](https://mlsanity.com/paper/2402.15258/) (2024)
- [Pop2Piano : Pop Audio-based Piano Cover Generation](https://mlsanity.com/paper/2211.00895/) (2022)
- [Audio-to-Score Alignment Using Deep Automatic Music Transcription](https://mlsanity.com/paper/2107.12854/) (2021)
- [Learning Frame Similarity using Siamese networks for Audio-to-Score Alignment](https://mlsanity.com/paper/2011.07546/) (2020)
- [Audio-to-score alignment of piano music using RNN-based automatic music transcription](https://mlsanity.com/paper/1711.04480/) (2017)

## Abstract (arXiv)

In this paper, we present a neural network approach for synchronizing audio recordings of human piano performances with their corresponding loosely aligned MIDI files. The task is addressed using a Convolutional Recurrent Neural Network (CRNN) architecture, which effectively captures spectral and temporal features by processing an unaligned piano roll and a spectrogram as inputs to estimate the aligned piano roll. To train the network, we create a dataset of piano pieces with augmented MIDI files that simulate common human timing errors. The proposed model achieves up to 20% higher alignment accuracy than the industry-standard Dynamic Time Warping (DTW) method across various tolerance windows. Furthermore, integrating DTW with the CRNN yields additional improvements, offering enhanced robustness and consistency. These findings demonstrate the potential of neural networks in advancing state-of-the-art MIDI-to-audio alignment.