mlsanityData to 7 Oct 2026

More than words: Advancements and challenges in speech recognition for singing

arXiv 2403.09298 ↗First posted 14 Mar 2024cs.SD cs.CL cs.IRcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper addresses the challenges and advancements in speech recognition for singing, a domain distinctly different from standard speech recognition. Singing encompasses unique challenges, including extensive pitch variations, diverse vocal styles, and background music interference. We explore key areas such as phoneme recognition, language identification in songs, keyword spotting, and full lyrics transcription. I will describe some of my own experiences when performing research on these tasks just as they were starting to gain traction, but will also show how recent developments in deep learning and large-scale datasets have propelled progress in this field. My goal is to illuminate the complexities of applying speech recognition to singing, evaluate current capabilities, and outline future research directions.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 59 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

14 of 59 references.

References · 14

  1. 2019DALI: a large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigmcited by 19
  2. 2019End-to-end Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Modelcited by 15
  3. 2023LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPTcited by 15
  4. 2022Transfer Learning of wav2vec 2.0 for Automatic Lyric Transcriptioncited by 11
  5. 2021Low Resource Audio-to-Lyrics Alignment From Polyphonic Music Recordingscited by 9
  6. 2021vocadito: A dataset of solo vocals with f0, note, and lyric annotationscited by 8
  7. 2020Automatic Lyrics Transcription using Dilated Convolutional Neural Networks with Self-Attentioncited by 7
  8. 2021MSTRE-Net: Multistreaming Acoustic Modeling for Automatic Lyrics Transcriptioncited by 7
  9. 2022Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Musiccited by 7
  10. 2019Acoustic Modeling for Automatic Lyrics-to-Audio Alignmentcited by 5
  11. 2021PDAugment: Data Augmentation by Pitch and Duration Adjustments for Automatic Lyrics Transcriptioncited by 5
  12. 2018Transcribing Lyrics From Commercial Song Audio: The First Step Towards Singing Content Processingcited by 2
2 more
  1. 2021Listen, Read, and Identify: Multimodal Singing Language Identification of Musiccited by 2
  2. 2021Singing Language Identification using a Deep Phonotactic Approachcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.