# Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond

arXiv: https://arxiv.org/abs/2304.05032 · HTML: https://mlsanity.com/paper/2304.05032/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 10 of 10)

- [Soft-DTW: a Differentiable Loss Function for Time-Series](https://mlsanity.com/paper/1703.01541/) (2017)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Onsets and Frames: Dual-Objective Piano Transcription](https://mlsanity.com/paper/1710.11153/) (2017)
- [D3TW: Discriminative Differentiable Dynamic Time Warping for Weakly Supervised Action Alignment and Segmentation](https://mlsanity.com/paper/1901.02598/) (2019)
- [Learning Features of Music from Scratch](https://mlsanity.com/paper/1611.09827/) (2016)
- [Representation Learning via Global Temporal Alignment and Cycle-Consistency](https://mlsanity.com/paper/2105.05217/) (2021)
- [On the Potential of Simple Framewise Approaches to Piano Transcription](https://mlsanity.com/paper/1612.05153/) (2016)
- [DeepNAG: Deep Non-Adversarial Gesture Generation](https://mlsanity.com/paper/2011.09149/) (2020)
- [Revisiting the Onsets and Frames Model with Additive Attention](https://mlsanity.com/paper/2104.06607/) (2021)
- [A Convolutional-Attentional Neural Framework for Structure-Aware Performance-Score Synchronization](https://mlsanity.com/paper/2204.08822/) (2022)

## Built on by (most cited first, 4 of 4)

- [Toward Fully Self-Supervised Multi-Pitch Estimation](https://mlsanity.com/paper/2402.15569/) (2024)
- [SCORE: Self-supervised Correspondence Fine-tuning for Improved Content Representations](https://mlsanity.com/paper/2403.06260/) (2024)
- [LASER: Learning by Aligning Self-supervised Representations of Speech for Improving Content-related Tasks](https://mlsanity.com/paper/2406.09153/) (2024)
- [ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio](https://mlsanity.com/paper/2408.00674/) (2024)

## Abstract (arXiv)

Many tasks in music information retrieval (MIR) involve weakly aligned data, where exact temporal correspondences are unknown. The connectionist temporal classification (CTC) loss is a standard technique to learn feature representations based on weakly aligned training data. However, CTC is limited to discrete-valued target sequences and can be difficult to extend to multi-label problems. In this article, we show how soft dynamic time warping (SoftDTW), a differentiable variant of classical DTW, can be used as an alternative to CTC. Using multi-pitch estimation as an example scenario, we show that SoftDTW yields results on par with a state-of-the-art multi-label extension of CTC. In addition to being more elegant in terms of its algorithmic formulation, SoftDTW naturally extends to real-valued target sequences.