mlsanityData to 7 Oct 2026

Towards Real-World Streaming Speech Translation for Code-Switched Speech

arXiv 2310.12648 ↗First posted 19 Oct 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Code-switching (CS), i.e. mixing different languages in a single sentence, is a common phenomenon in communication and can be challenging in many Natural Language Processing (NLP) settings. Previous studies on CS speech have shown promising results for end-to-end speech translation (ST), but have been limited to offline scenarios and to translation to one of the languages present in the source (textitmonolingual transcription). In this paper, we focus on two essential yet unexplored areas for real-world CS speech translation: streaming settings, and translation to a third language (i.e., a language not included in the source). To this end, we extend the Fisher and Miami test and validation datasets to include new targets in Spanish and German. Using this data, we train a model for both offline and streaming ST and we establish baseline results for the two settings mentioned earlier.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 17 of its 34 references that are in the map. First posted on arXiv 19 Oct 2023.

Builds on

17 of 34 references.

References · 17

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  6. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  7. 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 73
  8. 2021Are Multilingual Models Effective in Code-Switching?cited by 56
  9. 2021End-to-end Speech Translation via Cross-modal Progressive Trainingcited by 47
  10. 2020Code-switching pre-training for neural machine translationcited by 43
  11. 2018Low-Latency Neural Speech Translationcited by 35
  12. 2022End-to-End Speech Translation for Code Switched Speechcited by 26
5 more
  1. 2019End-to-End Code-Switching ASR for Low-Resourced Language Pairscited by 15
  2. 2022Code-Switching without Switching: Language Agnostic End-to-End Speech Translationcited by 13
  3. 2022ArzEn-ST: A Three-way Speech Translation Corpus for Code-Switched Egyptian Arabic - Englishcited by 13
  4. 2020Rnn-transducer with language bias for end-to-end Mandarin-English code-switching speech recognitioncited by 11
  5. 2021Streaming Models for Joint Speech Recognition and Translationcited by 10

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  2. 2025Beyond Monolingual Assumptions: A Survey of Code-Switched NLP in the Era of Large Language Models across Modalitiescited by 4
  3. 2024CoVoSwitch: Machine Translation of Synthetic Code-Switched Text Based on Intonation Unitscited by 1
  4. 2025Towards Fine-Grained Code-Switch Speech Translation with Semantic Space Alignmentcited by 1