mlsanityData to 14 Oct 2026

Data Quality Issues in Multilingual Speech Datasets: The Need for Sociolinguistic Awareness and Proactive Language Planning

arXiv 2506.17525 ↗First posted 21 Jun 2025cs.CL cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

Our quality audit for three widely used public multilingual speech datasets - Mozilla Common Voice 17.0, FLEURS, and Vox Populi - shows that in some languages, these datasets suffer from significant quality issues, which may obfuscate downstream evaluation results while creating an illusion of success. We divide these quality issues into two categories: micro-level and macro-level. We find that macro-level issues are more prevalent in less institutionalized, often under-resourced languages. We provide a case analysis of Taiwanese Southern Min (nan_tw) that highlights the need for proactive language planning (e.g. orthography prescriptions, dialect boundary definition) and enhanced data quality control in the dataset creation process. We conclude by proposing guidelines and recommendations to mitigate these issues in future dataset development, emphasizing the importance of sociolinguistic awareness and language planning principles. Furthermore, we encourage research into how this creation process itself can be leveraged as a tool for community-led language planning and revitalization.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 25 of its 58 references that are in the map. First posted on arXiv 21 Jun 2025.

Builds on

25 of 58 references.

References · 25

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  3. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  4. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 601
  5. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 560
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  8. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  9. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  10. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  11. 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 343
  12. 2021Quality at a Glance: An Audit of Web-Crawled Multilingual Datasetscited by 243
13 more
  1. 2020Hybrid Autoregressive Transducer (hat)cited by 108
  2. 2019Recognizing long-form speech using streaming end-to-end modelscited by 77
  3. 2020Rethinking Evaluation in ASR: Are Our Models Robust Enough?cited by 73
  4. 2021Sequence Length is a Domain: Length-based Overfitting in Transformer Modelscited by 35
  5. 2023Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentationcited by 31
  6. 2020RNN-T Models Fail to Generalize to Out-of-Domain Audio: Causes and Solutionscited by 26
  7. 2024Casablanca: Data and Models for Multidialectal Arabic Speech Recognitioncited by 20
  8. 2022ASR data augmentation in low-resource settings using cross-lingual multi-speaker TTS and cross-lingual voice conversioncited by 10
  9. 2023Lenient Evaluation of Japanese Speech Recognition: Modeling Naturally Occurring Spelling Inconsistencycited by 10
  10. 2022Language technology practitioners as language managers: arbitrating data bias and predictive bias in ASRcited by 7
  11. 2023ASR and Emotional Speech: A Word-Level Investigation of the Mutual Impact of Speech and Emotion Recognitioncited by 7
  12. 2023Developing a Multilingual Dataset and Evaluation Metrics for Code-Switching: A Focus on Hong Kong's Polylingual Dynamicscited by 3
  13. 2024DENOASR: Debiasing ASRs through Selective Denoisingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 9.

11 papers.

  1. 2025NADI 2025: The First Multidialectal Arabic Speech Processing Shared Taskcited by 7
  2. 2026Ethio-ASR: Joint Multilingual Speech Recognition and Language Identification for Ethiopian Languagescited by 2
  3. 2026Towards Comprehensive Semantic Speech Embeddings for Chinese Dialectscited by 1
  4. 2026Where Are We At with Automatic Speech Recognition for the Bambara Language?cited by 1
  5. 2026OpenBibleTTS: Large-Scale Speech Resources and TTS Models for Low-Resource Languagescited by 1
  6. 2025Balalaika: Data-Centric, Prosody-Aware Annotation Pipeline for Russian Speechcited by 0
  7. 2026AfroScope: A Framework for Studying the Linguistic Landscape of Africacited by 0
  8. 2026Variation is the Norm: Embracing Sociolinguistics in NLPcited by 0
  9. 2026VietPrism: A large-scale Vietnamese speech and deepfake corpus with diverse dialects and code-switchingcited by 0
  10. 2026SHAMS: An Audio-Grounded Pronunciation Benchmark for Levantine Arabiccited by 0
  11. 2026ELARSpeech: A 30K-Hour Dataset Across 675 Underrepresented Languages for Long-Tail Multilingual Speech Technologiescited by 0