# Acoustics-specific Piano Velocity Estimation

arXiv: https://arxiv.org/abs/2203.16294 · HTML: https://mlsanity.com/paper/2203.16294/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Cyclical Learning Rates for Training Neural Networks](https://mlsanity.com/paper/1506.01186/) (2015)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times](https://mlsanity.com/paper/2010.01815/) (2020)
- [Audio-to-Score Alignment Using Deep Automatic Music Transcription](https://mlsanity.com/paper/2107.12854/) (2021)
- [A Perceptual Measure for Evaluating the Resynthesis of Automatic Music Transcriptions](https://mlsanity.com/paper/2202.12257/) (2022)
- [ASMD: an automatic framework for compiling multimodal datasets with audio and scores](https://mlsanity.com/paper/2003.01958/) (2020)

## Built on by (most cited first, 1 of 1)

- [Variational Autoencoders for Anomaly Detection in Respiratory Sounds](https://mlsanity.com/paper/2208.03326/) (2022)

## Abstract (arXiv)

Motivated by the state-of-art psychological research, we note that a piano performance transcribed with existing Automatic Music Transcription (AMT) methods cannot be successfully resynthesized without affecting the artistic content of the performance. This is due to 1) the different mappings between MIDI parameters used by different instruments, and 2) the fact that musicians adapt their way of playing to the surrounding acoustic environment. To face this issue, we propose a methodology to build acoustics-specific AMT systems that are able to model the adaptations that musicians apply to convey their interpretation. Specifically, we train models tailored for virtual instruments in a modular architecture that takes as input an audio recording and the relative aligned music score, and outputs the acoustics-specific velocities of each note. We test different model shapes and show that the proposed methodology generally outperforms the usual AMT pipeline which does not consider specificities of the instrument and of the acoustic environment. Interestingly, such a methodology is extensible in a straightforward way since only slight efforts are required to train models for the inference of other piano parameters, such as pedaling.