# Automatic Lyrics Transcription using Dilated Convolutional Neural Networks with Self-Attention

arXiv: https://arxiv.org/abs/2007.06486 · HTML: https://mlsanity.com/paper/2007.06486/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2020.

## Built on by (most cited first, 7 of 7)

- [LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT](https://mlsanity.com/paper/2306.17103/) (2023)
- [MSTRE-Net: Multistreaming Acoustic Modeling for Automatic Lyrics Transcription](https://mlsanity.com/paper/2108.02625/) (2021)
- [Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Music](https://mlsanity.com/paper/2204.03307/) (2022)
- [Unsupervised Music Source Separation Using Differentiable Parametric Source Models](https://mlsanity.com/paper/2201.09592/) (2022)
- [Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studies](https://mlsanity.com/paper/2306.12714/) (2023)
- [More than words: Advancements and challenges in speech recognition for singing](https://mlsanity.com/paper/2403.09298/) (2024)
- [Elucidate Gender Fairness in Singing Voice Transcription](https://mlsanity.com/paper/2308.02898/) (2023)

## Abstract (arXiv)

Speech recognition is a well developed research field so that the current state of the art systems are being used in many applications in the software industry, yet as by today, there still does not exist such robust system for the recognition of words and sentences from singing voice. This paper proposes a complete pipeline for this task which may commonly be referred as automatic lyrics transcription (ALT). We have trained convolutional time-delay neural networks with self-attention on monophonic karaoke recordings using a sequence classification objective for building the acoustic model. The dataset used in this study, DAMP - Sing! 300x30x2 [1] is filtered to have songs with only English lyrics. Different language models are tested including MaxEnt and Recurrent Neural Networks based methods which are trained on the lyrics of pop songs in English. An in-depth analysis of the self-attention mechanism is held while tuning its context width and the number of attention heads. Using the best settings, our system achieves notable improvement to the state-of-the-art in ALT and provides a new baseline for the task.