mlsanityData to 13 Oct 2026

Enhancing Code-Switching ASR Leveraging Non-Peaky CTC Loss and Deep Language Posterior Injection

arXiv 2412.08651 ↗First posted 26 Nov 2024eess.AS cs.CL cs.LGcited by 0

On the map: Mixture-of-experts → Multimodal MoE models

Abstract (arXiv)

Code-switching-where multilingual speakers alternately switch between languages during conversations-still poses significant challenges to end-to-end (E2E) automatic speech recognition (ASR) systems due to phenomena of both acoustic and semantic confusion. This issue arises because ASR systems struggle to handle the rapid alternation of languages effectively, which often leads to significant performance degradation. Our main contributions are at least threefold: First, we incorporate language identification (LID) information into several intermediate layers of the encoder, aiming to enrich output embeddings with more detailed language information. Secondly, through the novel application of language boundary alignment loss, the subsequent ASR modules are enabled to more effectively utilize the knowledge of internal language posteriors. Third, we explore the feasibility of using language posteriors to facilitate deep interaction between shared encoder and language-specific encoders. Through comprehensive experiments on the SEAME corpus, we have verified that our proposed method outperforms the prior-art method, disentangle based mixture-of-experts (D-MoE), further enhancing the acuity of the encoder to languages.

Key facts. No citing papers in the map yet (data through 13 Oct 2026). Placed in Multimodal MoE models, one of 12 directions in Mixture-of-experts. Builds on 25 of its 31 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

25 of 31 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024An Effective Mixture-Of-Experts Approach For Code-Switching Speech Recognition Leveraging Encoder Disentanglement27 Feb

Other references · 24

  1. 2017AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baselinecited by 366
  2. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 185
  3. 2020Self-training and Pre-training are Complementary for Speech Recognitioncited by 99
  4. 2021Intermediate Loss Regularization for CTC-based Speech Recognitioncited by 95
  5. 2023FunASR: A Fundamental End-to-End Speech Recognition Toolkitcited by 93
  6. 2020Mask CTC: Non-Autoregressive End-to-End ASR with CTC and Mask Predictcited by 75
  7. 2021Relaxing the Conditional Independence Assumption of CTC-based ASR by Conditioning on Intermediate Predictionscited by 61
  8. 2020The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Resultscited by 43
  9. 2020Align-Refine: Non-Autoregressive Speech Recognition via Iterative Realignmentcited by 37
  10. 2021Why does CTC result in peaky behavior?cited by 32
  11. 2020Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognitioncited by 28
  12. 2022Reducing Language confusion for Code-switching Speech Recognition with Token-level Language Diarizationcited by 28
12 more
  1. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  2. 2022LAE: Language-Aware Encoder for Monolingual and Multilingual ASRcited by 21
  3. 2018Hierarchical Multi Task Learning With CTCcited by 20
  4. 2022Language-specific Characteristic Assistance for Code-switching Speech Recognitioncited by 20
  5. 2023Enhancing Code-switching Speech Recognition with Interactive Language Biasescited by 20
  6. 2022Towards Zero-Shot Code-Switched Speech Recognitioncited by 15
  7. 2024Less Peaky and More Accurate CTC Forced Alignment by Label Priorscited by 12
  8. 2021Minimum word error training for non-autoregressive Transformer-based code-switching ASRcited by 9
  9. 2022MMSpeech: Multi-modal Multi-task Encoder-Decoder Pre-training for Speech Recognitioncited by 9
  10. 2022Peak-First CTC: Reducing the Peak Latency of CTC Models by Applying Peak-First Regularizationcited by 5
  11. 2021Integrating Knowledge in End-to-End Automatic Speech Recognition for Mandarin-English Code-Switchingcited by 4
  12. 2024Low-resource speech recognition and dialect identification of Irish in a multi-task frameworkcited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignment
  2. 2023Language-Routing Mixture of Experts for Multilingual and Code-Switching Speech Recognition
  3. 2024CAMEL: Cross-Attention Enhanced Mixture-of-Experts and Language Bias for Code-Switching Speech Recognition
  4. 2026Speak in Context: Multilingual ASR with Speech Context Alignment via Contrastive Learning
  5. 2026NIM4-ASR: Towards Efficient, Robust, and Customizable Real-Time LLM-Based ASR
  6. 2023LAE-ST-MoE: Boosted Language-Aware Encoder Using Speech Translation Auxiliary Task for E2E Code-switching ASR
  7. 2024Enhancing Code-Switching Speech Recognition with LID-Based Collaborative Mixture of Experts Model
  8. 2024Boosting Code-Switching ASR with Mixture of Experts Enhanced Speech-Conditioned LLM