mlsanityData to 7 Oct 2026

Enhancing Multilingual LLM-based ASR with Mixture of Experts and Dynamic Downsampling

arXiv 2606.10439 ↗First posted 9 Jun 2026cs.SD cs.CL eess.AScited by 2

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

The rapid progress of large language models (LLMs) has opened up a new frontier for automatic speech recognition (ASR), making their effective integration a critical and challenging research direction. To this end, this work proposes a projector-based LLM-ASR framework targeting the key challenges of multilingual generalization and modality alignment. Our approach incorporates a Mixture of Experts (MoE) architecture to improve cross-lingual adaptability, and a Continuous Integrate-and-Fire (CIF) mechanism for dynamic downsampling and modality alignment. Experimental results show that the combination of these components yields substantial performance improvements, surpassing strong baseline models. The proposed method represents a step toward building more accurate, robust, and generalizable LLM-based ASR systems.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 21 of its 26 references that are in the map. First posted on arXiv 9 Jun 2026.

Builds on

21 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM24 Sep
  2. 2024SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASR26 Jun

Other references · 19

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,601
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 467
  6. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  8. 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 86
  9. 2023Connecting Speech Encoder and Large Language Model for ASRcited by 79
  10. 2019CIF: Continuous Integrate-and-Fire for End-to-End Speech Recognitioncited by 76
  11. 2019Correction of Automatic Speech Recognition with Transformer Sequence-to-sequence Modelcited by 52
  12. 2023Prompting Large Language Models for Zero-Shot Domain Adaptation in Speech Recognitioncited by 48
7 more
  1. 2024GigaSpeech 2: An Evolving, Large-Scale and Multi-domain ASR Corpus for Low-Resource Languages with Automated Crawling, Transcription and Refinementcited by 34
  2. 2023Revisiting Large Language Models as Zero-shot Relation Extractorscited by 29
  3. 2024ASR Error Correction using Large Language Modelscited by 23
  4. 2024Performance evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forwardcited by 17
  5. 2023Integrating Pre-Trained Speech and Language Models for End-to-End Speech Recognitioncited by 10
  6. 2025MOSA: Mixtures of Simple Adapters Outperform Monolithic Approaches in LLM-based Multilingual ASRcited by 4
  7. 2022Internal Language Model Estimation based Adaptive Language Model Fusion for Domain Adaptationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Language-Specialized Multi-Teacher On-Policy Distillation for Multilingual LLM-Based ASRcited by 1
  2. 2026Qwen-Audio-3.0-ASR Technical Reportcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  2. 2026Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
  3. 2024HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models
  4. 2025MLMA: Towards Multilingual ASR With Mamba-based Architectures
  5. 2024LA-RAG:Enhancing LLM-based ASR Accuracy with Retrieval-Augmented Generation
  6. 2026Rethinking Entropy Allocation in LLM-based ASR: Understanding the Dynamics between Speech Encoders and LLMs
  7. 2025PART: Progressive Alignment Representation Training for Multilingual Speech-To-Text with LLMs
  8. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognition