# Joint Estimation of Piano Dynamics and Metrical Structure with a Multi-task Multi-Scale Network

arXiv: https://arxiv.org/abs/2510.18190 · HTML: https://mlsanity.com/paper/2510.18190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 21 Oct 2025.

## Builds on (most cited first, 10 of 10)

- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [Music ControlNet: Multiple Time-varying Controls for Music Generation](https://mlsanity.com/paper/2311.07069/) (2023)
- [Beat this! Accurate beat tracking without DBN postprocessing](https://mlsanity.com/paper/2407.21658/) (2024)
- [A Data-Driven Analysis of Robust Automatic Piano Transcription](https://mlsanity.com/paper/2402.01424/) (2024)
- [End-to-End Real-World Polyphonic Piano Audio-to-Score Transcription with Hierarchical Decoding](https://mlsanity.com/paper/2405.13527/) (2024)
- [LLaQo: Towards a Query-Based Coach in Expressive Music Performance Assessment](https://mlsanity.com/paper/2409.08795/) (2024)
- [Frame-Level Multi-Label Playing Technique Detection Using Multi-Scale Network and Self-Attention Mechanism](https://mlsanity.com/paper/2303.13272/) (2023)
- [Automatic Estimation of Singing Voice Musical Dynamics](https://mlsanity.com/paper/2410.20540/) (2024)
- [Filling MIDI Velocity using U-Net Image Colorizer](https://mlsanity.com/paper/2508.07751/) (2025)

## Abstract (arXiv)

Estimating piano dynamic from audio recordings is a fundamental challenge in computational music analysis. In this paper, we propose an efficient multi-task network that jointly predicts dynamic levels, change points, beats, and downbeats from a shared latent representation. These four targets form the metrical structure of dynamics in the music score. Inspired by recent vocal dynamic research, we use a multi-scale network as the backbone, which takes Bark-scale specific loudness as the input feature. Compared to log-Mel as input, this reduces model size from 14.7 M to 0.5 M, enabling long sequential input. We use a 60-second audio length in audio segmentation, which doubled the length of beat tracking commonly used. Evaluated on the public MazurkaBL dataset, our model achieves state-of-the-art results across all tasks. This work sets a new benchmark for piano dynamic estimation and delivers a powerful and compact tool, paving the way for large-scale, resource-efficient analysis of musical expression.