mlsanityData to 7 Oct 2026

Mixture-of-Experts with Intermediate CTC Supervision for Accented Speech Recognition

arXiv 2602.01967 ↗First posted 2 Feb 2026cs.CL cs.AIcited by 0

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Accented speech remains a persistent challenge for automatic speech recognition (ASR), as most models are trained on data dominated by a few high-resource English varieties, leading to substantial performance degradation for other accents. Accent-agnostic approaches improve robustness yet struggle with heavily accented or unseen varieties, while accent-specific methods rely on limited and often noisy labels. We introduce Moe-Ctc, a Mixture-of-Experts architecture with intermediate CTC supervision that jointly promotes expert specialization and generalization. During training, accent-aware routing encourages experts to capture accent-specific patterns, which gradually transitions to label-free routing for inference. Each expert is equipped with its own CTC head to align routing with transcription quality, and a routing-augmented loss further stabilizes optimization. Experiments on the Mcv-Accent benchmark demonstrate consistent gains across both seen and unseen accents in low- and high-resource conditions, achieving up to 29.3% relative WER reduction over strong FastConformer baselines.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 27 of its 47 references that are in the map. First posted on arXiv 2 Feb 2026.

Builds on

27 of 47 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Mixture-of-Expert Conformer for Streaming Multilingual ASR25 May
  2. 2021SpeechMoE: Scaling to Large Acoustic Models with Dynamic Routing Mixture of Experts7 May

Other references · 25

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,014
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  7. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  8. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  9. 2023Fast Conformer with Linearly Scalable Attention for Efficient Speech Recognitioncited by 120
  10. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 95
  11. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  12. 2019Personalizing ASR for Dysarthric and Accented Speech with Limited Datacited by 45
13 more
  1. 2018Domain Adversarial Training for Accented Speech Recognitioncited by 40
  2. 2017Multi-Dialect Speech Recognition With A Single Sequence-To-Sequence Modelcited by 37
  3. 2021Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speechcited by 30
  4. 2024GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speechcited by 30
  5. 2020Learning Fast Adaptation on Cross-Accented Speech Recognitioncited by 28
  6. 2022Toward Fairness in Speech Recognition: Discovery and mitigation of performance disparitiescited by 27
  7. 2020REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabelingcited by 18
  8. 2021Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learningcited by 17
  9. 2021E2E-based Multi-task Learning Approach to Joint Speech and Accent Recognitioncited by 12
  10. 2021Accent-Robust Automatic Speech Recognition Using Supervised and Unsupervised Wav2vec Embeddingscited by 12
  11. 2022Layer-wise Fast Adaptation for End-to-End Multi-Accent Speech Recognitioncited by 12
  12. 2023Accented Speech Recognition With Accent-specific Codebookscited by 6
  13. 2024Improving Self-supervised Pre-training using Accent-Specific Codebookscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Decoder-only Conformer with Modality-aware Sparse Mixtures of Experts for ASR
  2. 2025Omni-Router: Sharing Routing Decisions in Sparse Mixture-of-Experts for Speech Recognition
  3. 2021Building a great multi-lingual teacher with sparsely-gated mixture of experts for speech recognition
  4. 2024U2++ MoE: Scaling 4.7x parameters with minimal impact on RTF
  5. 2024HDMoLE: Mixture of LoRA Experts with Hierarchical Routing and Dynamic Thresholds for Fine-Tuning LLM-based ASR Models
  6. 2025Accent Normalization Using Self-Supervised Discrete Tokens with Non-Parallel Data
  7. 2026TokAN: Accent Normalization Using Self-Supervised Speech Tokens
  8. 2026H-SAGE: Holistic Speaker-Aware Guided Experts for MoE-based Multi-Talker ASR