mlsanityData to 7 Oct 2026

Code-Switching without Switching: Language Agnostic End-to-End Speech Translation

arXiv 2210.01512 ↗First posted 4 Oct 2022cs.CL cs.LG cs.SDcited by 13

Not on the map. Why?

Abstract (arXiv)

We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increasingly used interchangeably during fluent speech. Such CS complicates traditional speech recognition and translation, as we must recognize which language was spoken first and then apply a language-dependent recognizer and subsequent translation component to generate the desired target language output. Such a pipeline introduces latency and errors. In this paper, we eliminate the need for that, by treating speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language. LAST delivers comparable recognition and speech translation accuracy in monolingual usage, while reducing latency and error rate considerably when CS is observed.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 13 of its 29 references that are in the map. First posted on arXiv 4 Oct 2022.

Builds on

13 of 29 references.

References · 13

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2016Learning without Forgettingcited by 2,465
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  7. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  8. 2019Very Deep Self-Attention Networks for End-to-End Speech Recognitioncited by 46
  9. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
  10. 2018Acoustic and Textual Data Augmentation for Improved ASR of Code-Switching Speechcited by 12
  11. 2020Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognitioncited by 10
  12. 2019LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognitioncited by 7
1 more
  1. 2022Reducing language context confusion for end-to-end code-switching automatic speech recognitioncited by 6

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 4; 2024: 2; 2025: 4.

13 papers.

Sort:

By area:

  1. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  2. 2023Recent Advances in Direct Speech-to-text Translationcited by 15
  3. 2022ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - Englishcited by 13
  4. 2023Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languagescited by 12
  5. 2025PIER: A Novel Metric for Evaluating What Matters in Code-Switchingcited by 11
  6. 2022LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducerscited by 10
  7. 2023End-to-End Evaluation for Low-Latency Simultaneous Speech Translationcited by 10
  8. 2025Weight Factorization and Centralization for Continual Learning in Speech Recognitioncited by 6
  9. 2022Generating Synthetic Speech from SpokenVocab for Speech Translationcited by 5
  10. 2023Towards Real-World Streaming Speech Translation for Code-Switched Speechcited by 4
  11. 2024CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleavingcited by 4
  12. 2025Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvementcited by 1
1 more
  1. 2025Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignmentcited by 1