mlsanityData to 7 Oct 2026

End-to-End Speech Translation for Code Switched Speech

arXiv 2204.05076 ↗First posted 11 Apr 2022cs.CL cs.SD eess.AScited by 26

Not on the map. Why?

Abstract (arXiv)

Code switching (CS) refers to the phenomenon of interchangeably using words and phrases from different languages. CS can pose significant accuracy challenges to NLP, due to the often monolingual nature of the underlying systems. In this work, we focus on CS in the context of English/Spanish conversations for the task of speech translation (ST), generating and evaluating both transcript and translation. To evaluate model performance on this task, we create a novel ST corpus derived from existing public data sets. We explore various ST architectures across two dimensions: cascaded (transcribe then translate) vs end-to-end (jointly transcribe and translate) and unidirectional (source -> target) vs bidirectional (source <-> target). We show that our ST architectures, and especially our bidirectional end-to-end architecture, perform well on CS speech, even when no CS training data is used.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 26 of its 61 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

26 of 61 references.

References · 26

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2020mT5: A massively multilingual pre-trained text-to-text transformercited by 1,813
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  5. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  6. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  7. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  8. 2020CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpuscited by 57
  9. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  10. 2018Tied Multitask Learning for Neural Speech Translationcited by 54
  11. 2019Code-Switching for Enhancing NMT with Pre-Specified Translationcited by 53
  12. 2020The ASRU 2019 Mandarin-English Code-Switching Speech Recognition Challenge: Open Datasets, Tracks, Methods and Resultscited by 43
14 more
  1. 2020Code-switching pre-training for neural machine translationcited by 43
  2. 2018On the End-to-End Solution to Mandarin-English Code-switching Speech Recognitioncited by 30
  3. 2021Can You Traducir This? Machine Translation for Code-Switched Inputcited by 21
  4. 2020Data Augmentation for End-to-end Code-switching Speech Recognitioncited by 20
  5. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  6. 2018Towards End-to-End Code-Switching Speech Recognitioncited by 17
  7. 2021Exploring Text-to-Text Transformers for English to Hinglish Machine Translation with Synthetic Code-Mixingcited by 16
  8. 2019End-to-End Code-Switching ASR for Low-Resourced Language Pairscited by 15
  9. 2020Consistent Transcription and Translation of Speechcited by 14
  10. 2018Building a Unified Code-Switching ASR System for South African Languagescited by 12
  11. 2019Subword-Level Language Identification for Intra-Word Code-Switchingcited by 12
  12. 2020Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognitioncited by 11
  13. 2021Streaming Models for Joint Speech Recognition and Translationcited by 10
  14. 2021Investigating Code-Mixed Modern Standard Arabic-Egyptian to English Machine Translationcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 9; 2024: 3; 2025: 9; 2026: 1.

26 papers.

Sort:

By area:

  1. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  2. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  3. 2022Code-Switching without Switching: Language Agnostic End-to-End Speech Translationcited by 13
  4. 2022ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - Englishcited by 13
  5. 2023Investigating Zero-Shot Generalizability on Mandarin-English Code-Switched ASR and Speech-to-text Translation of Recent Foundation Models with Self-Supervision and Weak Supervisioncited by 13
  6. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  7. 2025PIER: A Novel Metric for Evaluating What Matters in Code-Switchingcited by 11
  8. 2022LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducerscited by 10
  9. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  10. 2023Unified model for code-switching speech recognition and language identification based on a concatenated tokenizercited by 8
  11. 2025Weight Factorization and Centralization for Continual Learning in Speech Recognitioncited by 6
  12. 2022Generating Synthetic Speech from SpokenVocab for Speech Translationcited by 5
14 more
  1. 2023Towards Real-World Streaming Speech Translation for Code-Switched Speechcited by 4
  2. 2024CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleavingcited by 4
  3. 2025Code-Switching in End-to-End Automatic Speech Recognition: A Systematic Literature Reviewcited by 4
  4. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  5. 2023Towards a Deep Understanding of Multilingual End-to-End Speech Translationcited by 3
  6. 2025Low-resource Machine Translation for Code-switched Kazakh-Russian Language Paircited by 3
  7. 2025UniCoM: A Universal Code-Switching Speech Generatorcited by 2
  8. 2025Adapting Language Balance in Code-Switching Speechcited by 2
  9. 2023Simple yet Effective Code-Switching Language Identification with Multitask Pre-Training and Transfer Learningcited by 1
  10. 2023Combining Language Models For Specialized Domains: A Colorful Approachcited by 1
  11. 2024CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Unitscited by 1
  12. 2025Bayesian Low-Rank Factorization for Robust Model Adaptationcited by 1
  13. 2025Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignmentcited by 1
  14. 2026Adding Robust Code-Switching Capabilities to High Performance Multilingual ASRcited by 0