mlsanityData to 14 Oct 2026

Benchmarking Automatic Speech Recognition Tools for Iberian Languages

arXiv 2609.36920 ↗First posted 29 Sep 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Comprehensive evaluations of automatic speech recognition (ASR) for Iberian languages remain limited, and low-resource languages, biases, and efficiency trade-offs are underexplored. We benchmark eleven systems, ten open-weight models and one commercial API, across five Iberian languages (Basque, Catalan, Galician, Portuguese, Spanish), with German and Turkish as controls. Evaluation uses an 85-hour dataset covering read speech, broadcast media, and audiobooks, assessing accuracy and efficiency via word error rate (WER) and real-time factors (RTF/RTFx). Results show no single model dominates: accuracy, efficiency, and language coverage present clear trade-offs. Low-resource languages, especially Basque, degrade significantly, highlighting the role of training coverage. We observe consistent sex disparities across most systems, highlighting fairness challenges in multilingual ASR. Overall, the benchmark provides practical guidance for real-world model selection.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 21 of its 30 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

21 of 30 references.

References · 21

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2025Phi-4-Mini Technical Report: Compact yet Powerful Multimodal Language Models via Mixture-of-LoRAscited by 521
  6. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  7. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  8. 2017Word Embeddings Quantify 100 Years of Gender and Ethnic Stereotypescited by 253
  9. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 192
  10. 2026Qwen3-ASR Technical Reportcited by 118
  11. 2026Ministral 3cited by 114
  12. 2025Canary-1B-v2 & Parakeet-TDT-0.6B-v3: Efficient and High-Performance Models for Multilingual ASR and ASTcited by 60
9 more
  1. 2021Quantifying Bias in Automatic Speech Recognitioncited by 50
  2. 2025Voxtralcited by 43
  3. 2025Omnilingual ASR: Open-Source Multilingual Speech Recognition for 1600+ Languagescited by 34
  4. 2025Open ASR Leaderboard: Towards Reproducible and Transparent Multilingual and Long-Form Speech Recognition Evaluationcited by 23
  5. 2024Speech Robust Bench: A Robustness Benchmark For Speech Recognitioncited by 21
  6. 2021MediaSpeech: Multilanguage ASR Benchmark and Datasetcited by 17
  7. 2026Voxtral Realtimecited by 6
  8. 2025Whisper-LM: Improving ASR Models with Language Models for Low-Resource Languagescited by 5
  9. 2026WenetSpeech-Wu: Datasets, Benchmarks, and Models for a Unified Chinese Wu Dialect Speech Processing Ecosystemcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From Monolingual to Multilingual: Evaluating Mamba for ASR in South African Languages
  2. 2026Sometin Beta Pass Notin: Improving Multilingual ASR for Nigerian Languages via Knowledge Distillation
  3. 2025MLMA: Towards Multilingual ASR With Mamba-based Architectures
  4. 2024Anatomy of Industrial Scale Multilingual ASR
  5. 2024Efficient Compression of Multitask Multilingual Speech Models
  6. 2026ESCUCHA: A Spanish Speech Benchmark for Heterogeneous Acoustic Conditions
  7. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages
  8. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR