# Improving Lyrics Alignment through Joint Pitch Detection

arXiv: https://arxiv.org/abs/2202.01646 · HTML: https://mlsanity.com/paper/2202.01646/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Feb 2022.

## Built on by (most cited first, 6 of 6)

- [Contrastive Learning-Based Audio to Lyrics Alignment for Multiple Languages](https://mlsanity.com/paper/2306.07744/) (2023)
- [MERTech: Instrument Playing Technique Detection Using Self-Supervised Pretrained Model With Multi-Task Finetuning](https://mlsanity.com/paper/2310.09853/) (2023)
- [Multimodal Lyrics-Rhythm Matching](https://mlsanity.com/paper/2301.02732/) (2023)
- [ChordSync: Conformer-Based Alignment of Chord Annotations to Music Audio](https://mlsanity.com/paper/2408.00674/) (2024)
- [SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture](https://mlsanity.com/paper/2506.21478/) (2025)
- [A Real-Time Lyrics Alignment System Using Chroma And Phonetic Features For Classical Vocal Performance](https://mlsanity.com/paper/2401.09200/) (2024)

## Abstract (arXiv)

In recent years, the accuracy of automatic lyrics alignment methods has increased considerably. Yet, many current approaches employ frameworks designed for automatic speech recognition (ASR) and do not exploit properties specific to music. Pitch is one important musical attribute of singing voice but it is often ignored by current systems as the lyrics content is considered independent of the pitch. In practice, however, there is a temporal correlation between the two as note starts often correlate with phoneme starts. At the same time the pitch is usually annotated with high temporal accuracy in ground truth data while the timing of lyrics is often only available at the line (or word) level. In this paper, we propose a multi-task learning approach for lyrics alignment that incorporates pitch and thus can make use of a new source of highly accurate temporal information. Our results show that the accuracy of the alignment result is indeed improved by our approach. As an additional contribution, we show that integrating boundary detection in the forced-alignment algorithm reduces cross-line errors, which improves the accuracy even further.