# Stream-level Latency Evaluation for Simultaneous Machine Translation

arXiv: https://arxiv.org/abs/2104.08817 · HTML: https://mlsanity.com/paper/2104.08817/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 17 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 5 of 5)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Efficient Wait-k Models for Simultaneous Machine Translation](https://mlsanity.com/paper/2005.08595/) (2020)
- [Thinking Slow about Latency Evaluation for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.00048/) (2019)
- [ON-TRAC Consortium for End-to-End and Simultaneous Speech Translation Challenge Tasks at IWSLT 2020](https://mlsanity.com/paper/2005.11861/) (2020)

## Built on by (most cited first, 12 of 12)

- [StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection](https://mlsanity.com/paper/2406.06097/) (2024)
- [Average Token Delay: A Latency Metric for Simultaneous Translation](https://mlsanity.com/paper/2211.13173/) (2022)
- [Learning Optimal Policy for Simultaneous Machine Translation via Binary Search](https://mlsanity.com/paper/2305.12774/) (2023)
- [From Simultaneous to Streaming Machine Translation by Leveraging Streaming History](https://mlsanity.com/paper/2203.02459/) (2022)
- [Long-Form End-to-End Speech Translation via Latent Alignment Segmentation](https://mlsanity.com/paper/2309.11384/) (2023)
- [Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation](https://mlsanity.com/paper/2509.17349/) (2025)
- [Segmentation-Free Streaming Machine Translation](https://mlsanity.com/paper/2309.14823/) (2023)
- [Long-form Simultaneous Speech Translation: Thesis Proposal](https://mlsanity.com/paper/2310.11141/) (2023)
- [Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation](https://mlsanity.com/paper/2311.14353/) (2023)
- [Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters](https://mlsanity.com/paper/2407.13469/) (2024)
- [Assessing Latency in ASR Systems: A Methodological Perspective for Real-Time Use](https://mlsanity.com/paper/2409.05674/) (2024)
- [CA*: Addressing Evaluation Pitfalls in Computation-Aware Latency for Simultaneous Speech Translation](https://mlsanity.com/paper/2410.16011/) (2024)

## Abstract (arXiv)

Simultaneous machine translation has recently gained traction thanks to significant quality improvements and the advent of streaming applications. Simultaneous translation systems need to find a trade-off between translation quality and response time, and with this purpose multiple latency measures have been proposed. However, latency evaluations for simultaneous translation are estimated at the sentence level, not taking into account the sequential nature of a streaming scenario. Indeed, these sentence-level latency measures are not well suited for continuous stream translation resulting in figures that are not coherent with the simultaneous translation policy of the system being assessed. This work proposes a stream-level adaptation of the current latency measures based on a re-segmentation approach applied to the output translation, that is successfully evaluated on streaming conditions for a reference IWSLT task.