mlsanityData to 7 Oct 2026

Mandarin-English Code-switching Speech Recognition with Self-supervised Speech Representation Models

arXiv 2110.03504 ↗First posted 7 Oct 2021cs.CL eess.AScited by 9

Not on the map. Why?

Abstract (arXiv)

Code-switching (CS) is common in daily conversations where more than one language is used within a sentence. The difficulties of CS speech recognition lie in alternating languages and the lack of transcribed data. Therefore, this paper uses the recently successful self-supervised learning (SSL) methods to leverage many unlabeled speech data without CS. We show that hidden representations of SSL models offer frame-level language identity even if the models are trained with English speech only. Jointly training CTC and language identification modules with self-supervised speech representations improves CS speech recognition performance. Furthermore, using multilingual speech data for pre-training obtains the best CS speech recognition.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 34 references that are in the map. First posted on arXiv 7 Oct 2021.

Builds on

24 of 34 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  7. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  8. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 335
  9. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 293
  10. 2019Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoderscited by 196
  11. 2020TERA: Self-Supervised Learning of Transformer Encoder Representation for Speechcited by 186
  12. 2019An Unsupervised Autoregressive Model for Speech Representation Learningcited by 185
12 more
  1. 2020Unsupervised pretraining transfers well across languagescited by 100
  2. 2020Vector-Quantized Autoregressive Predictive Codingcited by 69
  3. 2020Non-Autoregressive Predictive Coding for Learning Speech Representations from Local Dependenciescited by 47
  4. 2018On the End-to-End Solution to Mandarin-English Code-switching Speech Recognitioncited by 30
  5. 2020Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognitioncited by 28
  6. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  7. 2018Code-switching Sentence Generation by Generative Adversarial Networks and its Application to Data Augmentationcited by 26
  8. 2020Data Augmentation for End-to-end Code-switching Speech Recognitioncited by 20
  9. 2020Improving Low Resource Code-switched ASR using Augmented Code-switched TTScited by 15
  10. 2020Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognitioncited by 11
  11. 2021Non-autoregressive Mandarin-English Code-switching Speech Recognitioncited by 8
  12. 2019Utterance-level end-to-end language identification using attention-based CNN-BLSTMcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 2; 2025: 1; 2026: 2.

9 papers.

  1. 2023Self-supervised Fine-tuning for Improved Content Representations by Speaker-invariant Clusteringcited by 21
  2. 2023Code-Switching Text Generation and Injection in Mandarin-English ASRcited by 10
  3. 2024Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decodingcited by 7
  4. 2023Adapting the adapters for code-switching in multilingual ASRcited by 6
  5. 2024Leave No Knowledge Behind During Knowledge Distillation: Towards Practical and Effective Knowledge Distillation for Code-Switching ASR Using Realistic Datacited by 5
  6. 2026ASPIRin: Action Space Projection for Interactivity-Optimized Reinforcement Learning in Full-Duplex Speech Language Modelscited by 3
  7. 2022Monolingual Recognizers Fusion for Code-switching Speech Recognitioncited by 2
  8. 2025Adapting Language Balance in Code-Switching Speechcited by 2
  9. 2026The Exact Worst-Case Tail Probability under Bounded Kurtosiscited by 0