mlsanityData to 14 Oct 2026

Efficient Multilingual ASR Finetuning via LoRA Language Experts

arXiv 2506.21555 ↗First posted 11 Jun 2025cs.CL cs.SD eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Recent advancements in deep learning have significantly enhanced multilingual automatic speech recognition (ASR) due to the development of advanced model architectures and available large-scale multilingual datasets. Despite that, multilingual ASR still suffers from the curse of multilinguality in that different languages tend to interfere with each other, making it difficult for the ASR model to identify multiple languages effectively while sharing model capacity across them. This paper proposes an efficient finetuning framework for customized multilingual ASR via prepared LoRA language experts based on Whisper. Through LoRA expert fusion or knowledge distillation, our approach achieves better recognition performance on target languages than standard fine-tuning methods. Experimental results demonstrate that the proposed models yield approximately 10% and 15% relative performance gains in language-aware and language-agnostic scenarios, respectively.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 23 of its 32 references that are in the map. First posted on arXiv 11 Jun 2025.

Builds on

23 of 32 references.

References · 23

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  5. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  6. 2015Listen, Attend and Spellcited by 543
  7. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  8. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 469
  9. 2023Scaling Speech Technology to 1,000+ Languagescited by 411
  10. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 373
  11. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 257
  12. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 231
11 more
  1. 2021Recent Advances in End-to-End Automatic Speech Recognitioncited by 140
  2. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 93
  3. 2024LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRcited by 36
  4. 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
  5. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognitioncited by 26
  6. 2023Multilingual DistilWhisper: Efficient Distillation of Multi-task Speech Models via Language-Specific Expertscited by 17
  7. 2024Towards Rehearsal-Free Multilingual ASR: A LoRA-based Case Study on Whispercited by 9
  8. 2024Continual Learning Optimizations for Auto-regressive Decoder of Multilingual ASR systemscited by 7
  9. 2024Weighted Cross-entropy for Low-Resource Languages in Multilingual Speech Recognitioncited by 6
  10. 2024A Parameter-efficient Language Extension Framework for Multilingual ASRcited by 3
  11. 2024Improving Multilingual ASR in the Wild Using Simple N-best Re-rankingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 5.

6 papers.

  1. 2025MLMA: Towards Multilingual ASR With Mamba-based Architecturescited by 1
  2. 2026A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASRcited by 1
  3. 2026Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languagescited by 1
  4. 2026Zipper-LoRA: Dynamic Parameter Decoupling for Speech-LLM based Multilingual Speech Recognitioncited by 1
  5. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0
  6. 2026CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognitioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Efficient Compression of Multitask Multilingual Speech Models
  2. 2026Enhancing Multilingual LLM-based ASR with Mixture of Experts and Dynamic Downsampling
  3. 2024LUPET: Incorporating Hierarchical Information Path into Multilingual ASR
  4. 2024HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models
  5. 2025The TEA-ASLP System for Multilingual Conversational Speech Recognition and Speech Diarization in MLC-SLM 2025 Challenge
  6. 2026Sometin Beta Pass Notin: Improving Multilingual ASR for Nigerian Languages via Knowledge Distillation
  7. 2024Anatomy of Industrial Scale Multilingual ASR
  8. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR