mlsanityData to 14 Oct 2026

MoLE : Mixture of Language Experts for Multi-Lingual Automatic Speech Recognition

arXiv 2302.13750 ↗First posted 27 Feb 2023eess.AS cs.CL cs.SDcited by 31

Not on the map. Why?

Abstract (arXiv)

Multi-lingual speech recognition aims to distinguish linguistic expressions in different languages and integrate acoustic processing simultaneously. In contrast, current multi-lingual speech recognition research follows a language-aware paradigm, mainly targeted to improve recognition performance rather than discriminate language characteristics. In this paper, we present a multi-lingual speech recognition network named Mixture-of-Language-Expert(MoLE), which digests speech in a variety of languages. Specifically, MoLE analyzes linguistic expression from input speech in arbitrary languages, activating a language-specific expert with a lightweight language tokenizer. The tokenizer not only activates experts, but also estimates the reliability of the activation. Based on the reliability, the activated expert and the language-agnostic expert are aggregated to represent language-conditioned embedding for efficient speech recognition. Our proposed model is evaluated in 5 languages scenario, and the experimental results show that our structure is advantageous on multi-lingual recognition, especially for speech in low-resource language.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 27 Feb 2023.

Builds on

18 of 28 references.

References · 18

  1. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  2. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  3. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  5. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  6. 2015Listen, Attend and Spellcited by 543
  7. 2015Attention-Based Models for Speech Recognitioncited by 401
  8. 2013Learning Factored Representations in a Deep Mixture of Expertscited by 278
  9. 2020Transformer Transducer: A Streamable Speech Recognition Model with Transformer Encoders and RNN-T Losscited by 235
  10. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 93
  11. 2021SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Expertscited by 48
  12. 2020Adapt-and-Adjust: Overcoming the Long-Tail Problem of Multilingual Speech Recognitioncited by 35
6 more
  1. 2019Meta Learning for End-to-End Low-Resource Speech Recognitioncited by 29
  2. 2021A Configurable Multilingual Model is All You Need to Recognize All Languagescited by 28
  3. 2021Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognitioncited by 16
  4. 2020Adversarial Meta Sampling for Multilingual Low-Resource Speech Recognitioncited by 15
  5. 2015Very Deep Multilingual Convolutional Neural Networks for LVCSRcited by 11
  6. 2021Multilingual Speech Recognition using Knowledge Transfer across Learning Processescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 13; 2025: 5; 2026: 3.

31 papers.

Sort:

By area:

  1. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognitioncited by 26
  2. 2024Efficiently Democratizing Medical LLMs for 50 Languages via a Mixture of Language Family Expertscited by 17
  3. 2023Adaptive Gating in Mixture-of-Experts based Language Modelscited by 16
  4. 2023LAE-ST-MoE: Boosted Language-Aware Encoder Using Speech Translation Auxiliary Task for E2E Code-switching ASRcited by 6
  5. 2024SC-MoE: Switch Conformer Mixture of Experts for Unified Streaming and Non-streaming Code-Switching ASRcited by 6
  6. 2023Confidence-based Ensembles of End-to-End Speech Recognition Modelscited by 4
  7. 2024On the Role of Discrete Representation in Sparse Mixture of Expertscited by 4
  8. 2025Sigmoid Self-Attention has Lower Sample Complexity than Softmax Self-Attention: A Mixture-of-Experts Perspectivecited by 4
  9. 2025Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognitioncited by 4
  10. 2023Condensing Multilingual Knowledge with Lightweight Language-Specific Modulescited by 3
  11. 2024SimSMoE: Solving Representational Collapse via Similarity Measurecited by 3
  12. 2023Big model only for hard audios: Sample dependent Whisper model selection for efficient inferencescited by 2
19 more
  1. 2023Self-supervised Adaptive Pre-training of Multilingual Speech Models for Language and Dialect Identificationcited by 2
  2. 2024Dynamic Language Group-Based MoE: Enhancing Code-Switching Speech Recognition with Hierarchical Routingcited by 2
  3. 2025BLR-MoE: Boosted Language-Routing Mixture of Experts for Domain-Robust Multilingual E2E ASRcited by 2
  4. 2024UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognitioncited by 1
  5. 2023Enhancing Multilingual Speech Recognition through Language Prompt Tuning and Frame-Level Language Adaptercited by 0
  6. 2023GMTR: Graph Matching Transformerscited by 0
  7. 2023DarkShot: Lighting Dark Images with Low-Compute and High-Qualitycited by 0
  8. 2024TRLS: A Time Series Representation Learning Framework via Spectrogram for Medical Signal Processingcited by 0
  9. 2024TEN-GUARD: Tensor Decomposition for Backdoor Attack Detection in Deep Neural Networkscited by 0
  10. 2024T-Pixel2Mesh: Combining Global and Local Transformer for 3D Mesh Generation from a Single Imagecited by 0
  11. 2024Strategic Arms with Side Communication Prevail Over Low-Regret MAB Algorithmscited by 0
  12. 2024Configurable Multilingual ASR with Speech Summary Representationscited by 0
  13. 2024Towards General Deepfake Detection with Dynamic Curriculumcited by 0
  14. 2024Attr-Int: A Simple and Effective Entity Alignment Framework for Heterogeneous Knowledge Graphscited by 0
  15. 2025On-the-fly Routing for Zero-shot MoE Speaker Adaptation of Speech Foundation Models for Dysarthric Speech Recognitioncited by 0
  16. 2025Mask6D: Masked Pose Priors For 6D Object Pose Estimationcited by 0
  17. 2026MiLorE-SSL: Scaling Multilingual Capabilities in Self-Supervised Models without Forgettingcited by 0
  18. 2026Lamer-SSL: Layer-aware Mixture of LoRA Experts for Continual Multilingual Expansion of Self-supervised Models without Forgettingcited by 0
  19. 2026MoLGE: Mixture of Language Group Experts for Efficient Scaling of Massively Multilingual Speech Recognitioncited by 0