mlsanityData to 7 Oct 2026

Genre-conditioned Acoustic Models for Automatic Lyrics Transcription of Polyphonic Music

arXiv 2204.03307 ↗First posted 7 Apr 2022cs.SD cs.AI eess.AScited by 7

Not on the map. Why?

Abstract (arXiv)

Lyrics transcription of polyphonic music is challenging not only because the singing vocals are corrupted by the background music, but also because the background music and the singing style vary across music genres, such as pop, metal, and hip hop, which affects lyrics intelligibility of the song in different ways. In this work, we propose to transcribe the lyrics of polyphonic music using a novel genre-conditioned network. The proposed network adopts pre-trained model parameters, and incorporates the genre adapters between layers to capture different genre peculiarities for lyrics-genre pairs, thereby only requiring lightweight genre-specific parameters for training. Our experiments show that the proposed genre-conditioned network outperforms the existing lyrics transcription systems.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 14 of its 38 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

14 of 38 references.

References · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2016Layer Normalizationcited by 5,579
  4. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2021Lightweight Adapter Tuning for Multilingual Speech Translationcited by 53
  7. 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
  8. 2018End-to-end Speech Recognition with Word-based RNN Language Modelscited by 23
  9. 2019DALI: a large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigmcited by 19
  10. 2019End-to-end Lyrics Alignment for Polyphonic Music Using an Audio-to-Character Recognition Modelcited by 15
  11. 2021Low Resource Audio-to-Lyrics Alignment From Polyphonic Music Recordingscited by 9
  12. 2020Automatic Lyrics Transcription using Dilated Convolutional Neural Networks with Self-Attentioncited by 7
2 more
  1. 2019Acoustic Modeling for Automatic Lyrics-to-Audio Alignmentcited by 5
  2. 2019Automatic Lyrics Alignment and Transcription in Polyphonic Music: Does Background Music Help?cited by 2

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 4; 2024: 1.

7 papers.

  1. 2022Generate, Discriminate and Contrast: A Semi-Supervised Sentence Representation Learning Frameworkcited by 18
  2. 2023LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPTcited by 15
  3. 2023Dynamic Transformers Provide a False Sense of Efficiencycited by 11
  4. 2022token2vec: A Joint Self-Supervised Pre-training Framework Using Unpaired Speech and Textcited by 3
  5. 2023Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studiescited by 1
  6. 2024More than words: Advancements and challenges in speech recognition for singingcited by 1
  7. 2023Elucidate Gender Fairness in Singing Voice Transcriptioncited by 0