# Jam-ALT: A Formatting-Aware Lyrics Transcription Benchmark

arXiv: https://arxiv.org/abs/2311.13987 · HTML: https://mlsanity.com/paper/2311.13987/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 23 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 11 of 11)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [Is 42 the Answer to Everything in Subtitling-oriented Speech Translation?](https://mlsanity.com/paper/2006.01080/) (2020)
- [End-to-end Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Model](https://mlsanity.com/paper/1902.06797/) (2019)
- [LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT](https://mlsanity.com/paper/2306.17103/) (2023)
- [Capitalization and Punctuation Restoration: a Survey](https://mlsanity.com/paper/2111.10746/) (2021)
- [Transfer Learning of wav2vec 2.0 for Automatic Lyric Transcription](https://mlsanity.com/paper/2207.09747/) (2022)
- [Contrastive Learning-Based Audio to Lyrics Alignment for Multiple Languages](https://mlsanity.com/paper/2306.07744/) (2023)
- [Low Resource Audio-to-Lyrics Alignment From Polyphonic Music Recordings](https://mlsanity.com/paper/2102.09202/) (2021)
- [MSTRE-Net: Multistreaming Acoustic Modeling for Automatic Lyrics Transcription](https://mlsanity.com/paper/2108.02625/) (2021)

## Built on by (most cited first, 2 of 2)

- [The Esethu Framework: Reimagining Sustainable Dataset Governance and Curation for Low-Resource Languages](https://mlsanity.com/paper/2502.15916/) (2025)
- [Roadmap towards Superhuman Speech Understanding using Large Language Models](https://mlsanity.com/paper/2410.13268/) (2024)

## Abstract (arXiv)

Current automatic lyrics transcription (ALT) benchmarks focus exclusively on word content and ignore the finer nuances of written lyrics including formatting and punctuation, which leads to a potential misalignment with the creative products of musicians and songwriters as well as listeners' experiences. For example, line breaks are important in conveying information about rhythm, emotional emphasis, rhyme, and high-level structure. To address this issue, we introduce Jam-ALT, a new lyrics transcription benchmark based on the JamendoLyrics dataset. Our contribution is twofold. Firstly, a complete revision of the transcripts, geared specifically towards ALT evaluation by following a newly created annotation guide that unifies the music industry's guidelines, covering aspects such as punctuation, line breaks, spelling, background vocals, and non-word sounds. Secondly, a suite of evaluation metrics designed, unlike the traditional word error rate, to capture such phenomena. We hope that the proposed benchmark contributes to the ALT task, enabling more precise and reliable assessments of transcription systems and enhancing the user experience in lyrics applications such as subtitle renderings for live captioning or karaoke.