# Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcription

arXiv: https://arxiv.org/abs/2404.09466 · HTML: https://mlsanity.com/paper/2404.09466/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Exploring Plain Vision Transformer Backbones for Object Detection](https://mlsanity.com/paper/2203.16527/) (2022)
- [Axial Attention in Multidimensional Transformers](https://mlsanity.com/paper/1912.12180/) (2019)
- [AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradients](https://mlsanity.com/paper/2010.07468/) (2020)
- [ReZero is All You Need: Fast Convergence at Large Depth](https://mlsanity.com/paper/2003.04887/) (2020)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detection](https://mlsanity.com/paper/2106.00666/) (2021)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [Learnable Fourier Features for Multi-Dimensional Spatial Positional Encoding](https://mlsanity.com/paper/2106.02795/) (2021)
- [High-resolution Piano Transcription with Pedals by Regressing Onset and Offset Times](https://mlsanity.com/paper/2010.01815/) (2020)
- [Sequence-to-Sequence Piano Transcription with Transformers](https://mlsanity.com/paper/2107.09142/) (2021)
- [The continuous Bernoulli: fixing a pervasive error in variational autoencoders](https://mlsanity.com/paper/1907.06845/) (2019)
- [Music Source Separation with Band-Split RoPE Transformer](https://mlsanity.com/paper/2309.02612/) (2023)
- [Automatic Piano Transcription with Hierarchical Frequency-Time Transformer](https://mlsanity.com/paper/2307.04305/) (2023)
- [Deep Polyphonic ADSR Piano Note Transcription](https://mlsanity.com/paper/1906.09165/) (2019)
- [SepTr: Separable Transformer for Audio Spectrogram Processing](https://mlsanity.com/paper/2203.09581/) (2022)
- [Polyphonic Piano Transcription Using Autoregressive Multi-State Note Model](https://mlsanity.com/paper/2010.01104/) (2020)

## Built on by (most cited first, 14 of 14)

- [Scaling Self-Supervised Representation Learning for Symbolic Piano Performance](https://mlsanity.com/paper/2506.23869/) (2025)
- [PianoVAM: A Multimodal Piano Performance Dataset](https://mlsanity.com/paper/2509.08800/) (2025)
- [D3RM: A Discrete Denoising Diffusion Refinement Model for Piano Transcription](https://mlsanity.com/paper/2501.05068/) (2025)
- [SyMuPe: Affective and Controllable Symbolic Music Performance](https://mlsanity.com/paper/2511.03425/) (2025)
- [Noise-to-Notes: Diffusion-based Generation and Refinement for Automatic Drum Transcription](https://mlsanity.com/paper/2509.21739/) (2025)
- [VioPTT: Violin Technique-Aware Transcription from Synthetic Data Augmentation](https://mlsanity.com/paper/2509.23759/) (2025)
- [PianoCoRe: Combined and Refined Piano MIDI Dataset](https://mlsanity.com/paper/2605.06627/) (2026)
- [HARP 2.0: Expanding Hosted, Asynchronous, Remote Processing for Deep Learning in the DAW](https://mlsanity.com/paper/2503.02977/) (2025)
- [High-Resolution Sustain Pedal Depth Estimation from Piano Audio Across Room Acoustics](https://mlsanity.com/paper/2507.04230/) (2025)
- [RUMAA: Repeat-Aware Unified Music Audio Analysis for Score-Performance Alignment, Transcription, and Mistake Detection](https://mlsanity.com/paper/2507.12175/) (2025)
- [Rubato: Transcribing Piano Music with Timestamps](https://mlsanity.com/paper/2605.24291/) (2026)
- [PianoKontext: Expressive Performance Rendering from Deadpan Context](https://mlsanity.com/paper/2606.12282/) (2026)
- [Learning to Predict Performance-induced Emotion Differences in Classical Piano Music](https://mlsanity.com/paper/2607.28876/) (2026)
- [Multi-Task Multi-Frame Visual Piano Transcription](https://mlsanity.com/paper/2608.03419/) (2026)

## Abstract (arXiv)

The neural semi-Markov Conditional Random Field (semi-CRF) framework has demonstrated promise for event-based piano transcription. In this framework, all events (notes or pedals) are represented as closed time intervals tied to specific event types. The neural semi-CRF approach requires an interval scoring matrix that assigns a score for every candidate interval. However, designing an efficient and expressive architecture for scoring intervals is not trivial. This paper introduces a simple method for scoring intervals using scaled inner product operations that resemble how attention scoring is done in transformers. We show theoretically that, due to the special structure from encoding the non-overlapping intervals, under a mild condition, the inner product operations are expressive enough to represent an ideal scoring matrix that can yield the correct transcription result. We then demonstrate that an encoder-only structured non-hierarchical transformer backbone, operating only on a low-time-resolution feature map, is capable of transcribing piano notes and pedals with high accuracy and time precision. The experiment shows that our approach achieves the new state-of-the-art performance across all subtasks in terms of the F1 measure on the Maestro dataset.