mlsanityData to 7 Oct 2026

GLAD: Global-Local Aware Dynamic Mixture-of-Experts for Multi-Talker ASR

arXiv 2509.13093 ↗First posted 16 Sep 2025cs.SDcited by 1

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

End-to-end multi-talker automatic speech recognition (MTASR) faces significant challenges in accurately transcribing overlapping speech. A critical bottleneck is that speaker-specific acoustic characteristics, which are essential for distinguishing overlapping speech, are often diluted in deep network layers. To address this, we propose the Global-Local Aware Dynamic Mixture-of-Experts (GLAD) architecture. GLAD introduces a novel routing mechanism that dynamically fuses speaker-aware global context with fine-grained local acoustic details to adaptively guide expert selection. Experiments on the LibriSpeechMix and CH109 datasets demonstrate that GLAD significantly outperforms existing Serialized Output Training (SOT)-based MTASR approaches, exhibiting exceptional robustness in challenging, high-overlap scenarios. To the best of our knowledge, this is the first work to apply a global-local fusion MoE strategy to MTASR.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 25 of its 30 references that are in the map. First posted on arXiv 16 Sep 2025.

Builds on

25 of 30 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognition17 Dec
  2. 2024HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models30 Sep
  3. 2024Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM24 Sep

Other references · 22

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 189
  7. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 97
  8. 2020Serialized Output Training for End-to-End Overlapped Speech Recognitioncited by 87
  9. 2020End-to-End Multi-speaker Speech Recognition with Transformercited by 42
  10. 2020Joint Speaker Counting, Speech Recognition, and Speaker Identification for Overlapped Speech of Any Number of Speakerscited by 41
  11. 2021End-to-End Speaker-Attributed ASR with Transformercited by 35
  12. 2017Recognizing Multi-talker Speech with Permutation Invariant Trainingcited by 33
10 more
  1. 2020Streaming end-to-end multi-talker speech recognitioncited by 29
  2. 2024Large Language Model Can Transcribe Speech in Multi-Talker Scenarios with Versatile Instructionscited by 17
  3. 2024Empowering Whisper as a Joint Multi-Talker and Target-Talker Speech Recognition Systemcited by 11
  4. 2024Cross-Speaker Encoding Network for Multi-Talker Speech Recognitioncited by 9
  5. 2025Speaker Targeting via Self-Speaker Adaptation for Multi-talker ASRcited by 7
  6. 2024SA-SOT: Speaker-Aware Serialized Output Training for Multi-Talker ASRcited by 6
  7. 2024Serialized Output Training by Learned Dominancecited by 5
  8. 2024Disentangling Speakers in Multi-Talker Speech Recognition with Speaker-Aware CTCcited by 4
  9. 2025Streaming Sortformer: Speaker Cache-Based Online Speaker Diarization with Arrival-Time Orderingcited by 4
  10. 2025Speaker-Distinguishable CTC: Learning Speaker Distinction Using CTC for Multi-Talker Speech Recognitioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026H-SAGE: Holistic Speaker-Aware Guided Experts for MoE-based Multi-Talker ASRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
  2. 2026Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR
  3. 2026Dynamic Multi-Expert Projectors with Stabilized Routing for Multilingual Speech Recognition
  4. 2025Beyond Hard Sharing: Efficient Multi-Task Speech-to-Text Modeling with Supervised Mixture of Experts
  5. 2023Mixture-of-Expert Conformer for Streaming Multilingual ASR
  6. 2024UME: Upcycling Mixture-of-Experts for Scalable and Efficient Automatic Speech Recognition
  7. 2024U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF
  8. 2021Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition