mlsanityData to 7 Oct 2026

Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition

arXiv 2307.05956 ↗First posted 12 Jul 2023cs.SD eess.AScited by 26

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Multilingual speech recognition for both monolingual and code-switching speech is a challenging task. Recently, based on the Mixture of Experts (MoE), many works have made good progress in multilingual and code-switching ASR, but present huge computational complexity with the increase of supported languages. In this work, we propose a computation-efficient network named Language-Routing Mixture of Experts (LR-MoE) for multilingual and code-switching ASR. LR-MoE extracts language-specific representations through the Mixture of Language Experts (MLE), which is guided to learn by a frame-wise language routing mechanism. The weight-shared frame-level language identification (LID) network is jointly trained as the shared pre-router of each MoE layer. Experiments show that the proposed method significantly improves multilingual and code-switching speech recognition performances over baseline with comparable computational efficiency.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 27 of its 39 references that are in the map. First posted on arXiv 12 Jul 2023.

Builds on

27 of 39 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition10 Dec
  2. 2021SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts7 May

Other references · 25

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  7. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  8. 2015Listen, Attend and Spellcited by 542
  9. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  10. 2015Attention-Based Models for Speech Recognitioncited by 400
  11. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 364
  12. 2016Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learningcited by 254
13 more
  1. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 92
  2. 2017Advances in Joint CTC-Attention based End-to-End Speech Recognition with a Deep CNN Encoder and RNN-LMcited by 43
  3. 2020The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Resultscited by 43
  4. 2023MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognitioncited by 31
  5. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  6. 2022LAE: Language-Aware Encoder for Monolingual and Multilingual ASRcited by 21
  7. 2022Language-specific Characteristic Assistance for Code-switching Speech Recognitioncited by 20
  8. 2021Acoustic word embeddings for zero-resource languages using self-supervised contrastive learning and multilingual adaptationcited by 19
  9. 2018Towards End-to-End Code-Switching Speech Recognitioncited by 17
  10. 2021Joint Modeling of Code-Switched and Monolingual ASR via Conditional Factorizationcited by 17
  11. 2022Leveraging Phone Mask Training for Phonetic-Reduction-Robust E2E Uyghur Speech Recognitioncited by 6
  12. 2021Multilingual Speech Recognition using Knowledge Transfer across Learning Processescited by 4
  13. 2021PM-MMUT: Boosted Phone-Mask Data Augmentation using Multi-Modeling Unit Training for Phonetic-Reduction-Robust E2E Speech Recognitioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 11; 2025: 9; 2026: 4.

26 papers.

Sort:

By area:

  1. 2024LoRA-Whisper: Parameter-Efficient and Extensible Multilingual ASRcited by 35
  2. 2023From Google Gemini to OpenAI Q* (Q-Star): A Survey of Reshaping the Generative Artificial Intelligence (AI) Research Landscapecited by 18
  3. 2023LAE-ST-MoE: Boosted Language-Aware Encoder Using Speech Translation Auxiliary Task for E2E Code-switching ASRcited by 6
  4. 2024SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASRcited by 6
  5. 2025Low-Rank and Sparse Model Merging for Multi-Lingual Speech Recognition and Translationcited by 6
  6. 2024U2++ MoE: Scaling 4.7x parameters with minimal impact on RTFcited by 5
  7. 2025Can we train ASR systems on Code-switch without real code-switch data? Case study for Singapore's languagescited by 5
  8. 2025Efficient Multilingual ASR Finetuning via LoRA Language Expertscited by 5
  9. 2024The Second DISPLACE Challenge : DIarization of SPeaker and LAnguage in Conversational Environmentscited by 4
  10. 2024On the Role of Discrete Representation in Sparse Mixture of Expertscited by 4
  11. 2025MoHAVE: Mixture of Hierarchical Audio-Visual Experts for Robust Speech Recognitioncited by 4
  12. 2024SimSMoE: Solving Representational Collapse via Similarity Measurecited by 3
14 more
  1. 2024Dynamic Language Group-Based MoE: Enhancing Code-Switching Speech Recognition with Hierarchical Routingcited by 2
  2. 2024Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Modelcited by 2
  3. 2024TouchASP: Elastic Automatic Speech Perception that Everyone Can Touchcited by 2
  4. 2025BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASRcited by 2
  5. 2025Language-Aware Prompt Tuning for Parameter-Efficient Seamless Language Expansion in Multilingual ASRcited by 2
  6. 2025Adapting Language Balance in Code-Switching Speechcited by 2
  7. 2024UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognitioncited by 1
  8. 2025SHNU Multilingual Conversational Speech Recognition System for INTERSPEECH 2025 MLC-SLM Challengecited by 1
  9. 2026A Language-Agnostic Hierarchical LoRA-MoE Architecture for CTC-based Multilingual ASRcited by 1
  10. 2026Language-Aware Distillation for Multilingual Instruction-Following Speech LLMs with ASR-Only Supervisioncited by 1
  11. 2024Configurable Multilingual ASR with Speech Summary Representationscited by 0
  12. 2025Joint Learning using Mixture-of-Expert-Based Representation for Speech Enhancement and Robust Emotion Recognitioncited by 0
  13. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0
  14. 2026RouteRec: Behavior-Guided Sparse Routing for Sequential Recommendationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM
  2. 2024An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement
  3. 2024CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognition
  4. 2026Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
  5. 2025Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
  6. 2024Enhancing Code-Switching ASR Leveraging Non-Peaky CTC Loss and Deep Language Posterior Injection
  7. 2023Mixture-of-Expert Conformer for Streaming Multilingual ASR
  8. 2023BA-MoE: Boundary-Aware Mixture-of-Experts Adapter for Code-Switching Speech Recognition