# Code-Switching without Switching: Language Agnostic End-to-End Speech Translation

arXiv: https://arxiv.org/abs/2210.01512 · HTML: https://mlsanity.com/paper/2210.01512/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 4 Oct 2022.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019)
- [Very Deep Self-Attention Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1904.13377/) (2019)
- [End-to-End Speech Translation for Code Switched Speech](https://mlsanity.com/paper/2204.05076/) (2022)
- [Acoustic and Textual Data Augmentation for Improved ASR of Code-Switching Speech](https://mlsanity.com/paper/1807.10945/) (2018)
- [Learning to Recognize Code-switched Speech Without Forgetting Monolingual Speech Recognition](https://mlsanity.com/paper/2006.00782/) (2020)
- [LibriVoxDeEn: A Corpus for German-to-English Speech Translation and German Speech Recognition](https://mlsanity.com/paper/1910.07924/) (2019)
- [Reducing language context confusion for end-to-end code-switching automatic speech recognition](https://mlsanity.com/paper/2201.12155/) (2022)

## Built on by (most cited first, 13 of 13)

- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Recent Advances in Direct Speech-to-text Translation](https://mlsanity.com/paper/2306.11646/) (2023)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [Zero Resource Code-switched Speech Benchmark Using Speech Utterance Pairs For Multiple Spoken Languages](https://mlsanity.com/paper/2310.03018/) (2023)
- [PIER: A Novel Metric for Evaluating What Matters in Code-Switching](https://mlsanity.com/paper/2501.09512/) (2025)
- [LAMASSU: Streaming Language-Agnostic Multilingual Speech Recognition and Translation Using Neural Transducers](https://mlsanity.com/paper/2211.02809/) (2022)
- [End-to-End Evaluation for Low-Latency Simultaneous Speech Translation](https://mlsanity.com/paper/2308.03415/) (2023)
- [Weight Factorization and Centralization for Continual Learning in Speech Recognition](https://mlsanity.com/paper/2506.16574/) (2025)
- [Generating Synthetic Speech from SpokenVocab for Speech Translation](https://mlsanity.com/paper/2210.08174/) (2022)
- [Towards Real-World Streaming Speech Translation for Code-Switched Speech](https://mlsanity.com/paper/2310.12648/) (2023)
- [CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleaving](https://mlsanity.com/paper/2406.10993/) (2024)
- [Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement](https://mlsanity.com/paper/2506.16580/) (2025)
- [Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignment](https://mlsanity.com/paper/2511.10670/) (2025)

## Abstract (arXiv)

We propose a) a Language Agnostic end-to-end Speech Translation model (LAST), and b) a data augmentation strategy to increase code-switching (CS) performance. With increasing globalization, multiple languages are increasingly used interchangeably during fluent speech. Such CS complicates traditional speech recognition and translation, as we must recognize which language was spoken first and then apply a language-dependent recognizer and subsequent translation component to generate the desired target language output. Such a pipeline introduces latency and errors. In this paper, we eliminate the need for that, by treating speech recognition and translation as one unified end-to-end speech translation problem. By training LAST with both input languages, we decode speech into one target language, regardless of the input language. LAST delivers comparable recognition and speech translation accuracy in monolingual usage, while reducing latency and error rate considerably when CS is observed.