# Towards Real-World Streaming Speech Translation for Code-Switched Speech

arXiv: https://arxiv.org/abs/2310.12648 · HTML: https://mlsanity.com/paper/2310.12648/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 34 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019)
- [Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation](https://mlsanity.com/paper/1611.04558/) (2016)
- [CoVoST 2 and Massively Multilingual Speech-to-Text Translation](https://mlsanity.com/paper/2007.10310/) (2020)
- [Are Multilingual Models Effective in Code-Switching?](https://mlsanity.com/paper/2103.13309/) (2021)
- [End-to-end Speech Translation via Cross-modal Progressive Training](https://mlsanity.com/paper/2104.10380/) (2021)
- [Code-switching pre-training for neural machine translation](https://mlsanity.com/paper/2009.08088/) (2020)
- [Low-Latency Neural Speech Translation](https://mlsanity.com/paper/1808.00491/) (2018)
- [End-to-End Speech Translation for Code Switched Speech](https://mlsanity.com/paper/2204.05076/) (2022)
- [End-to-End Code-Switching ASR for Low-Resourced Language Pairs](https://mlsanity.com/paper/1909.12681/) (2019)
- [Code-Switching without Switching: Language Agnostic End-to-End Speech Translation](https://mlsanity.com/paper/2210.01512/) (2022)
- [ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - English](https://mlsanity.com/paper/2211.12000/) (2022)
- [Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognition](https://mlsanity.com/paper/2002.08126/) (2020)
- [Streaming Models for Joint Speech Recognition and Translation](https://mlsanity.com/paper/2101.09149/) (2021)

## Built on by (most cited first, 4 of 4)

- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalities](https://mlsanity.com/paper/2510.07037/) (2025)
- [CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Units](https://mlsanity.com/paper/2407.14295/) (2024)
- [Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignment](https://mlsanity.com/paper/2511.10670/) (2025)

## Abstract (arXiv)

Code-switching (CS), i.e. mixing different languages in a single sentence, is a common phenomenon in communication and can be challenging in many Natural Language Processing (NLP) settings. Previous studies on CS speech have shown promising results for end-to-end speech translation (ST), but have been limited to offline scenarios and to translation to one of the languages present in the source (textitmonolingual transcription). In this paper, we focus on two essential yet unexplored areas for real-world CS speech translation: streaming settings, and translation to a third language (i.e., a language not included in the source). To this end, we extend the Fisher and Miami test and validation datasets to include new targets in Spanish and German. Using this data, we train a model for both offline and streaming ST and we establish baseline results for the two settings mentioned earlier.