Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
On the map: Mixture-of-experts → Multimodal MoE models
Abstract (arXiv)
Recent advances in LLM-based ASR connect frozen speech encoders with Large Language Models (LLMs) via lightweight projectors. While effective in monolingual settings, a single projector struggles to capture the diverse acoustic-to-semantic mappings required for multilingual ASR. To address this, we propose SMEAR-MoE, a stabilized Mixture-of-Experts projector that ensures dense gradient flow to all experts, preventing expert collapse while enabling cross-lingual sharing. We systematically compare monolithic, static multi-projector, and dynamic MoE designs across four Indic languages (Hindi, Marathi, Tamil, Telugu). Our SMEAR-MoE achieves strong performance, delivering upto a 7.6% relative WER reduction over the single-projector baseline, while maintaining comparable runtime efficiency. Analysis of expert routing further shows linguistically meaningful specialization, with related languages sharing experts. These results demonstrate that stable multi-expert projectors are key to scalable and robust multilingual ASR.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 18 of its 18 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
18 of 18 references.
Closest ancestors · 2
Same direction, newest first
- 2025Scaling and Enhancing LLM-based AVSR: A Sparse Mixture of Projectors Approach20 May
- 2024HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models30 Sep
Other references · 16
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
- 2023Scaling Speech Technology to 1,000+ Languagescited by 401
- 2023Soft Merging of Experts with Adaptive Routingcited by 88
- 2024An Embarrassingly Simple Approach for LLM with Strong ASR Capacitycited by 86
- 2023Connecting Speech Encoder and Large Language Model for ASRcited by 79
- 2021Multilingual and code-switching ASR challenges for low resource Indian languagescited by 50
- 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 39
- 2022IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languagescited by 38
4 more
- 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
- 2025Granite-speech: open-source speech-aware LLMs with strong English ASR capabilitiescited by 26
- 2023Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASRcited by 21
- 2025MOSA: Mixtures of Simple Adapters Outperform Monolithic Approaches in LLM-based Multilingual ASRcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Enhancing Multilingual LLM-based ASR with Mixture of Experts and Dynamic Downsampling
- 2023Mixture-of-Expert Conformer for Streaming Multilingual ASR
- 2021Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition
- 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition
- 2024LUPET: Incorporating Hierarchical Information Path into Multilingual ASR
- 2024Dynamic Language Group-Based MoE: Enhancing Code-Switching Speech Recognition with Hierarchical Routing
- 2025GLAD: Global-Local Aware Dynamic Mixture-of-Experts for Multi-Talker ASR
- 2025MLMA: Towards Multilingual ASR With Mamba-based Architectures