# Online Symbolic Music Alignment with Offline Reinforcement Learning

arXiv: https://arxiv.org/abs/2401.00466 · HTML: https://mlsanity.com/paper/2401.00466/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 34 references that are in the map.
- First posted on arXiv 31 Dec 2023.

## Builds on (most cited first, 6 of 6)

- [Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problems](https://mlsanity.com/paper/2005.01643/) (2020)
- [Learning to Listen, Read, and Follow: Score Following as a Reinforcement Learning Game](https://mlsanity.com/paper/1807.06391/) (2018)
- [The ACCompanion: Combining Reactivity, Robustness, and Musical Expressivity in an Automatic Piano Accompanist](https://mlsanity.com/paper/2304.12939/) (2023)
- [Exact, Parallelizable Dynamic Time Warping Alignment with Linear Memory](https://mlsanity.com/paper/2008.02734/) (2020)
- [Outer-Product Hidden Markov Model and Polyphonic MIDI Score Following](https://mlsanity.com/paper/1404.2313/) (2014)
- [A Stochastic Temporal Model of Polyphonic MIDI Performance with Ornaments](https://mlsanity.com/paper/1404.2314/) (2014)

## Built on by (most cited first, 5 of 5)

- [TheGlueNote: Learned Representations for Robust and Flexible Note Alignment](https://mlsanity.com/paper/2408.04309/) (2024)
- [PianoCoRe: Combined and Refined Piano MIDI Dataset](https://mlsanity.com/paper/2605.06627/) (2026)
- [PianoKontext: Expressive Performance Rendering from Deadpan Context](https://mlsanity.com/paper/2606.12282/) (2026)
- [Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances](https://mlsanity.com/paper/2607.27909/) (2026)
- [TUTTI: Toward generalizable audio-to-score transcription via fully synthesized data](https://mlsanity.com/paper/2609.00640/) (2026)

## Abstract (arXiv)

Symbolic Music Alignment is the process of matching performed MIDI notes to corresponding score notes. In this paper, we introduce a reinforcement learning (RL)-based online symbolic music alignment technique. The RL agent - an attention-based neural network - iteratively estimates the current score position from local score and performance contexts. For this symbolic alignment task, environment states can be sampled exhaustively and the reward is dense, rendering a formulation as a simplified offline RL problem straightforward. We evaluate the trained agent in three ways. First, in its capacity to identify correct score positions for sampled test contexts; second, as the core technique of a complete algorithm for symbolic online note-wise alignment; and finally, as a real-time symbolic score follower. We further investigate the pitch-based score and performance representations used as the agent's inputs. To this end, we develop a second model, a two-step Dynamic Time Warping (DTW)-based offline alignment algorithm leveraging the same input representation. The proposed model outperforms a state-of-the-art reference model of offline symbolic music alignment.