mlsanityData to 14 Oct 2026

Optimizing Bilingual Neural Transducer with Synthetic Code-switching Text Generation

arXiv 2210.12214 ↗First posted 21 Oct 2022cs.SD cs.CL eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Code-switching describes the practice of using more than one language in the same sentence. In this study, we investigate how to optimize a neural transducer based bilingual automatic speech recognition (ASR) model for code-switching speech. Focusing on the scenario where the ASR model is trained without supervised code-switching data, we found that semi-supervised training and synthetic code-switched data can improve the bilingual ASR system on code-switching speech. We analyze how each of the neural transducer's encoders contributes towards code-switching performance by measuring encoder-specific recall values, and evaluate our English/Mandarin system on the ASCEND data set. Our final system achieves 25% mixed error rate (MER) on the ASCEND English/Mandarin code-switching test set -- reducing the MER by 2.1% absolute compared to the previous literature -- while maintaining good accuracy on the monolingual test sets.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 23 of its 37 references that are in the map. First posted on arXiv 21 Oct 2022.

Builds on

23 of 37 references.

References · 23

  1. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  2. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,863
  3. 2015Listen, Attend and Spellcited by 543
  4. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  5. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
  6. 2020Pushing the Limits of Semi-Supervised Learning for Automatic Speech Recognitioncited by 187
  7. 2019Self-Training for End-to-End Speech Recognitioncited by 129
  8. 2019End-to-end ASR: from Supervised to Semi-Supervised Learning with Modern Architecturescited by 120
  9. 2020Hybrid Autoregressive Transducer (hat)cited by 108
  10. 2016Towards better decoding and language model integration in sequence to sequence modelscited by 100
  11. 2019Large-Scale Multilingual Speech Recognition with a Streaming End-to-End Modelcited by 93
  12. 2020Internal Language Model Estimation for Domain-Adaptive End-to-End Speech Recognitioncited by 80
11 more
  1. 2020A Density Ratio Approach to Language Model Fusion in End-To-End Automatic Speech Recognitioncited by 73
  2. 2019Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentencescited by 55
  3. 2022Pruned RNN-T for fast, memory-efficient ASR trainingcited by 46
  4. 2021ASCEND: A Spontaneous Chinese-English Dataset for Code-switching in Multi-turn Conversationcited by 44
  5. 2021From Machine Translation to Code-Switching: Generating High-Quality Code-Switched Textcited by 29
  6. 2020Transformer-Transducers for Code-Switched Speech Recognitioncited by 27
  7. 2021Pseudo-Labeling for Massively Multilingual Speech Recognitioncited by 24
  8. 2020Improved Neural Language Model Fusion for Streaming Recurrent Neural Network Transducercited by 22
  9. 2022Language-specific Characteristic Assistance for Code-switching Speech Recognitioncited by 20
  10. 2019SNDCNN: Self-normalizing deep CNNs with scaled exponential linear units for speech recognitioncited by 13
  11. 2022Bilingual End-to-End ASR with Byte-Level Subwordscited by 4

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 0; 2025: 1.

5 papers.

  1. 2023Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalizationcited by 43
  2. 2023Approximate Nearest Neighbour Phrase Mining for Contextual Speech Recognitioncited by 8
  3. 2023LAE-ST-MoE: Boosted Language-Aware Encoder Using Speech Translation Auxiliary Task for E2E Code-switching ASRcited by 6
  4. 2023Cross-lingual Knowledge Transfer and Iterative Pseudo-labeling for Low-Resource Speech Recognition with Transducerscited by 2
  5. 2025SAGE: Spliced-Audio Generated Data for Enhancing Foundational Models in Low-Resource Arabic-English Code-Switched Speech Recognitioncited by 2