# End-to-End Evaluation for Low-Latency Simultaneous Speech Translation

arXiv: https://arxiv.org/abs/2308.03415 · HTML: https://mlsanity.com/paper/2308.03415/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 27 references that are in the map.
- First posted on arXiv 7 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021)
- [Multilingual Denoising Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2001.08210/) (2020)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [The Multilingual TEDx Corpus for Speech Recognition and Translation](https://mlsanity.com/paper/2102.01757/) (2021)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoders](https://mlsanity.com/paper/2106.13736/) (2021)
- [Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection](https://mlsanity.com/paper/2005.11185/) (2020)
- [Low-Latency Neural Speech Translation](https://mlsanity.com/paper/1808.00491/) (2018)
- [CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022](https://mlsanity.com/paper/2204.06028/) (2022)
- [Code-Switching without Switching: Language Agnostic End-to-End Speech Translation](https://mlsanity.com/paper/2210.01512/) (2022)
- [KIT's Multilingual Speech Translation System for IWSLT 2023](https://mlsanity.com/paper/2306.05320/) (2023)

## Built on by (most cited first, 10 of 10)

- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Graph Canvas for Controllable 3D Scene Generation](https://mlsanity.com/paper/2412.00091/) (2024)
- [PIER: A Novel Metric for Evaluating What Matters in Code-Switching](https://mlsanity.com/paper/2501.09512/) (2025)
- [Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation](https://mlsanity.com/paper/2509.17349/) (2025)
- [Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems](https://mlsanity.com/paper/2512.17648/) (2025)
- [Titanic Calling: Low Bandwidth Video Conference from the Titanic Wreck](https://mlsanity.com/paper/2410.11434/) (2024)
- [Bayesian Low-Rank Factorization for Robust Model Adaptation](https://mlsanity.com/paper/2510.18723/) (2025)
- [Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026](https://mlsanity.com/paper/2606.04730/) (2026)
- [From Slides to Chatbots: Enhancing Large Language Models with University Course Materials](https://mlsanity.com/paper/2510.22272/) (2025)
- [Asynchronous Pipeline Parallelism for Real-Time Multilingual Lip Synchronization in Video Communication Systems](https://mlsanity.com/paper/2512.18318/) (2025)

## Abstract (arXiv)

The challenge of low-latency speech translation has recently draw significant interest in the research community as shown by several publications and shared tasks. Therefore, it is essential to evaluate these different approaches in realistic scenarios. However, currently only specific aspects of the systems are evaluated and often it is not possible to compare different approaches. In this work, we propose the first framework to perform and evaluate the various aspects of low-latency speech translation under realistic conditions. The evaluation is carried out in an end-to-end fashion. This includes the segmentation of the audio as well as the run-time of the different components. Secondly, we compare different approaches to low-latency speech translation using this framework. We evaluate models with the option to revise the output as well as methods with fixed output. Furthermore, we directly compare state-of-the-art cascaded as well as end-to-end systems. Finally, the framework allows to automatically evaluate the translation quality as well as latency and also provides a web interface to show the low-latency model outputs to the user.