# ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio

arXiv: https://arxiv.org/abs/2408.00674 · HTML: https://mlsanity.com/paper/2408.00674/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 54 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Self-supervised Learning with Random-projection Quantizer for Speech Recognition](https://mlsanity.com/paper/2202.01855/) (2022)
- [A Foundation Model for Music Informatics](https://mlsanity.com/paper/2311.03318/) (2023)
- [End-to-end Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Model](https://mlsanity.com/paper/1902.06797/) (2019)
- [Audio-to-Score Alignment Using Deep Automatic Music Transcription](https://mlsanity.com/paper/2107.12854/) (2021)
- [Chord Label Personalization through Deep Learning of Integrated Harmonic Interval-based Representations](https://mlsanity.com/paper/1706.09552/) (2017)
- [Improving Lyrics Alignment through Joint Pitch Detection](https://mlsanity.com/paper/2202.01646/) (2022)
- [Soft Dynamic Time Warping for Multi-Pitch Estimation and Beyond](https://mlsanity.com/paper/2304.05032/) (2023)
- [Exploiting Time-Frequency Conformers for Music Audio Enhancement](https://mlsanity.com/paper/2308.12599/) (2023)
- [Stabilizing Training with Soft Dynamic Time Warping: A Case Study for Pitch Class Estimation with Weakly Aligned Targets](https://mlsanity.com/paper/2308.05429/) (2023)

## Built on by (most cited first, 2 of 2)

- [From Discord to Harmony: Decomposed Consonance-based Training for Improved Audio Chord Estimation](https://mlsanity.com/paper/2509.01588/) (2025)
- [ChordFormer: A Conformer-Based Architecture for Large-Vocabulary Audio Chord Recognition](https://mlsanity.com/paper/2502.11840/) (2025)

## Abstract (arXiv)

In the Western music tradition, chords are the main constituent components of harmony, a fundamental dimension of music. Despite its relevance for several Music Information Retrieval (MIR) tasks, chord-annotated audio datasets are limited and need more diversity. One way to improve those resources is to leverage the large number of chord annotations available online, but this requires aligning them with music audio. However, existing audio-to-score alignment techniques, which typically rely on Dynamic Time Warping (DTW), fail to address this challenge, as they require weakly aligned data for precise synchronisation. In this paper, we introduce ChordSync, a novel conformer-based model designed to seamlessly align chord annotations with audio, eliminating the need for weak alignment. We also provide a pre-trained model and a user-friendly library, enabling users to synchronise chord annotations with audio tracks effortlessly. In this way, ChordSync creates opportunities for harnessing crowd-sourced chord data for MIR, especially in audio chord estimation, thereby facilitating the generation of novel datasets. Additionally, our system extends its utility to music education, enhancing music learning experiences by providing accurately aligned annotations, thus enabling learners to engage in synchronised musical practices.