# Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning

arXiv: https://arxiv.org/abs/2506.17525 · HTML: https://mlsanity.com/paper/2506.17525/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 58 references that are in the map.
- First posted on arXiv 21 Jun 2025.

## Builds on (most cited first, 25 of 25)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation](https://mlsanity.com/paper/2106.03193/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech](https://mlsanity.com/paper/2205.12446/) (2022)
- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [Scaling Speech Technology to 1,000+ Languages](https://mlsanity.com/paper/2305.13516/) (2023)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [Hybrid Autoregressive Transducer (hat)](https://mlsanity.com/paper/2003.07705/) (2020)
- [Recognizing long-form speech using streaming end-to-end models](https://mlsanity.com/paper/1910.11455/) (2019)
- [Rethinking Evaluation in ASR: Are Our Models Robust Enough?](https://mlsanity.com/paper/2010.11745/) (2020)
- [Sequence Length is a Domain: Length-based Overfitting in Transformer Models](https://mlsanity.com/paper/2109.07276/) (2021)
- [Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation](https://mlsanity.com/paper/2305.10951/) (2023)
- [RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutions](https://mlsanity.com/paper/2005.03271/) (2020)
- [Casablanca: Data and Models for Multidialectal Arabic Speech Recognition](https://mlsanity.com/paper/2410.04527/) (2024)
- [ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversion](https://mlsanity.com/paper/2204.00618/) (2022)
- [Lenient Evaluation of Japanese Speech Recognition: Modeling Naturally Occurring Spelling Inconsistency](https://mlsanity.com/paper/2306.04530/) (2023)
- [Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASR](https://mlsanity.com/paper/2202.12603/) (2022)
- [ASR and Emotional Speech: A Word-Level Investigation of the Mutual Impact of Speech and Emotion Recognition](https://mlsanity.com/paper/2305.16065/) (2023)
- [Developing a Multilingual Dataset and Evaluation Metrics for Code-Switching: A Focus on Hong Kong's Polylingual Dynamics](https://mlsanity.com/paper/2310.17953/) (2023)
- [DENOASR: Debiasing ASRs through Selective Denoising](https://mlsanity.com/paper/2410.16712/) (2024)

## Built on by (most cited first, 11 of 11)

- [NADI 2025: The First Multidialectal Arabic Speech Processing Shared Task](https://mlsanity.com/paper/2509.02038/) (2025)
- [Ethio-ASR: Joint Multilingual Speech Recognition and Language Identification for Ethiopian Languages](https://mlsanity.com/paper/2603.23654/) (2026)
- [Towards Comprehensive Semantic Speech Embeddings for Chinese Dialects](https://mlsanity.com/paper/2601.07274/) (2026)
- [Where Are We At with Automatic Speech Recognition for the Bambara Language?](https://mlsanity.com/paper/2602.09785/) (2026)
- [OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languages](https://mlsanity.com/paper/2606.09553/) (2026)
- [Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speech](https://mlsanity.com/paper/2507.13563/) (2025)
- [AfroScope: A Framework for Studying the Linguistic Landscape of Africa](https://mlsanity.com/paper/2601.13346/) (2026)
- [Variation is the Norm: Embracing Sociolinguistics in NLP](https://mlsanity.com/paper/2603.24222/) (2026)
- [VietPrism: A large-scale Vietnamese speech and deepfake corpus with diverse dialects and code-switching](https://mlsanity.com/paper/2609.30005/) (2026)
- [SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabic](https://mlsanity.com/paper/2610.01427/) (2026)
- [ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologies](https://mlsanity.com/paper/2610.19388/) (2026)

## Abstract (arXiv)

Our quality audit for three widely used public multilingual speech datasets - Mozilla Common Voice 17.0, FLEURS, and Vox Populi - shows that in some languages, these datasets suffer from significant quality issues, which may obfuscate downstream evaluation results while creating an illusion of success. We divide these quality issues into two categories: micro-level and macro-level. We find that macro-level issues are more prevalent in less institutionalized, often under-resourced languages. We provide a case analysis of Taiwanese Southern Min (nan_tw) that highlights the need for proactive language planning (e.g. orthography prescriptions, dialect boundary definition) and enhanced data quality control in the dataset creation process. We conclude by proposing guidelines and recommendations to mitigate these issues in future dataset development, emphasizing the importance of sociolinguistic awareness and language planning principles. Furthermore, we encourage research into how this creation process itself can be leveraged as a tool for community-led language planning and revitalization.