# PhoWhisper: Automatic Speech Recognition for Vietnamese

arXiv: https://arxiv.org/abs/2406.02555 · HTML: https://mlsanity.com/paper/2406.02555/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 9 references that are in the map.
- First posted on arXiv 27 Mar 2024.

## Builds on (most cited first, 6 of 6)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [SeamlessM4T: Massively Multilingual & Multimodal Machine Translation](https://mlsanity.com/paper/2308.11596/) (2023)

## Built on by (most cited first, 13 of 13)

- [Do Prompts Really Prompt? Exploring the Prompt Understanding Capability of Whisper](https://mlsanity.com/paper/2406.05806/) (2024)
- [Multi-Dialect Vietnamese: Task, Dataset, Baseline Models and Challenges](https://mlsanity.com/paper/2410.03458/) (2024)
- [Zero-Shot Text-to-Speech for Vietnamese](https://mlsanity.com/paper/2506.01322/) (2025)
- [ViMedCSS: A Vietnamese Medical Code-Switching Speech Dataset & Benchmark](https://mlsanity.com/paper/2602.12911/) (2026)
- [Flavors of Moonshine: Tiny Specialized ASR Models for Edge Devices](https://mlsanity.com/paper/2509.02523/) (2025)
- [ViCocktail: Automated Multi-Modal Data Collection for Vietnamese Audio-Visual Speech Recognition](https://mlsanity.com/paper/2506.04635/) (2025)
- [Whisper based Cross-Lingual Phoneme Recognition between Vietnamese and English](https://mlsanity.com/paper/2508.19270/) (2025)
- [Phonetic Modeling of Dialectal Variation in Vietnamese Speech](https://mlsanity.com/paper/2605.24451/) (2026)
- [ViToSA: Audio-Based Toxic Spans Detection on Vietnamese Speech Utterances](https://mlsanity.com/paper/2506.00636/) (2025)
- [TSPC: A Two-Stage Phoneme-Centric Architecture for code-switching Vietnamese-English Speech Recognition](https://mlsanity.com/paper/2509.05983/) (2025)
- [VietLyrics: A Large-Scale Dataset and Models for Vietnamese Automatic Lyrics Transcription](https://mlsanity.com/paper/2510.22295/) (2025)
- [Vietnamese Automatic Speech Recognition: A Revisit](https://mlsanity.com/paper/2603.14779/) (2026)
- [PiDA: Phonetically-Informed Data Augmentation for Robust Vietnamese Speech Translation](https://mlsanity.com/paper/2606.12911/) (2026)

## Abstract (arXiv)

We introduce PhoWhisper in five versions for Vietnamese automatic speech recognition. PhoWhisper's robustness is achieved through fine-tuning the Whisper model on an 844-hour dataset that encompasses diverse Vietnamese accents. Our experimental study demonstrates state-of-the-art performances of PhoWhisper on benchmark Vietnamese ASR datasets. We have open-sourced PhoWhisper at: https://github.com/VinAIResearch/PhoWhisper