# VietLyrics: A Large-Scale Dataset and Models for Vietnamese Automatic Lyrics Transcription

arXiv: https://arxiv.org/abs/2510.22295 · HTML: https://mlsanity.com/paper/2510.22295/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 19 references that are in the map.
- First posted on arXiv 25 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [DALI: a large Dataset of synchronized Audio, LyrIcs and notes, automatically created using teacher-student machine learning paradigm](https://mlsanity.com/paper/1906.10606/) (2019)
- [LyricWhiz: Robust Multilingual Zero-shot Lyrics Transcription by Whispering to ChatGPT](https://mlsanity.com/paper/2306.17103/) (2023)
- [PhoWhisper: Automatic Speech Recognition for Vietnamese](https://mlsanity.com/paper/2406.02555/) (2024)
- [Contrastive Learning-Based Audio to Lyrics Alignment for Multiple Languages](https://mlsanity.com/paper/2306.07744/) (2023)
- [Lyrics Transcription for Humans: A Readability-Aware Benchmark](https://mlsanity.com/paper/2408.06370/) (2024)

## Abstract (arXiv)

Automatic Lyrics Transcription (ALT) for Vietnamese music presents unique challenges due to its tonal complexity and dialectal variations, but remains largely unexplored due to the lack of a dedicated dataset. Therefore, we curated the first large-scale Vietnamese ALT dataset (VietLyrics), comprising 647 hours of songs with line-level aligned lyrics and metadata to address these issues. Our evaluation of current ASRbased approaches reveal significant limitations, including frequent transcription errors and hallucinations in non-vocal segments. To improve performance, we fine-tuned Whisper models on the VietLyrics dataset, achieving superior results compared to existing multilingual ALT systems, including LyricWhiz. We publicly release VietLyrics and our models, aiming to advance Vietnamese music computing research while demonstrating the potential of this approach for ALT in low-resource language and music.