# Multi-Dialect Vietnamese: Task, Dataset, Baseline Models and Challenges

arXiv: https://arxiv.org/abs/2410.03458 · HTML: https://mlsanity.com/paper/2410.03458/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [The People's Speech: A Large-Scale Diverse English Speech Recognition Dataset for Commercial Usage](https://mlsanity.com/paper/2111.09344/) (2021)
- [QASR: QCRI Aljazeera Speech Resource -- A Large Scale Annotated Arabic Speech Corpus](https://mlsanity.com/paper/2106.13000/) (2021)
- [STT4SG-350: A Speech Corpus for All Swiss German Dialect Regions](https://mlsanity.com/paper/2305.18855/) (2023)
- [PhoWhisper: Automatic Speech Recognition for Vietnamese](https://mlsanity.com/paper/2406.02555/) (2024)
- [A Deep Neural Network for Audio Classification with a Classifier Attention Mechanism](https://mlsanity.com/paper/2006.09815/) (2020)

## Built on by (most cited first, 6 of 6)

- [WenetSpeech-Chuan: A Large-Scale Sichuanese Corpus with Rich Annotation for Dialectal Speech Processing](https://mlsanity.com/paper/2509.18004/) (2025)
- [MoRoVoc: A Large Dataset for Geographical Variation Identification of the Spoken Romanian Language](https://mlsanity.com/paper/2509.16781/) (2025)
- [Phonetic Modeling of Dialectal Variation in Vietnamese Speech](https://mlsanity.com/paper/2605.24451/) (2026)
- [Advancing Uto-Aztecan Language Technologies: A Case Study on the Endangered Comanche Language](https://mlsanity.com/paper/2505.18159/) (2025)
- [Vietnamese Automatic Speech Recognition: A Revisit](https://mlsanity.com/paper/2603.14779/) (2026)
- [Beyond Phones: Structured Phonemic Modeling for Vietnamese Automatic Speech Recognition](https://mlsanity.com/paper/2605.27874/) (2026)

## Abstract (arXiv)

Vietnamese, a low-resource language, is typically categorized into three primary dialect groups that belong to Northern, Central, and Southern Vietnam. However, each province within these regions exhibits its own distinct pronunciation variations. Despite the existence of various speech recognition datasets, none of them has provided a fine-grained classification of the 63 dialects specific to individual provinces of Vietnam. To address this gap, we introduce Vietnamese Multi-Dialect (ViMD) dataset, a novel comprehensive dataset capturing the rich diversity of 63 provincial dialects spoken across Vietnam. Our dataset comprises 102.56 hours of audio, consisting of approximately 19,000 utterances, and the associated transcripts contain over 1.2 million words. To provide benchmarks and simultaneously demonstrate the challenges of our dataset, we fine-tune state-of-the-art pre-trained models for two downstream tasks: (1) Dialect identification and (2) Speech recognition. The empirical results suggest two implications including the influence of geographical factors on dialects, and the constraints of current approaches in speech recognition tasks involving multi-dialect speech data. Our dataset is available for research purposes.