# Thinking Slow about Latency Evaluation for Simultaneous Machine Translation

arXiv: https://arxiv.org/abs/1906.00048 · HTML: https://mlsanity.com/paper/1906.00048/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 May 2019.

## Built on by (most cited first, 28 of 28)

- [Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation](https://mlsanity.com/paper/2206.05807/) (2022)
- [TransLLaMa: LLM-based Simultaneous Translation System](https://mlsanity.com/paper/2402.04636/) (2024)
- [Attention as a Guide for Simultaneous Speech Translation](https://mlsanity.com/paper/2212.07850/) (2022)
- [Learning When to Translate for Streaming Speech](https://mlsanity.com/paper/2109.07368/) (2021)
- [CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022](https://mlsanity.com/paper/2204.06028/) (2022)
- [Does Simultaneous Speech Translation need Simultaneous Models?](https://mlsanity.com/paper/2204.03783/) (2022)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Exploring Continuous Integrate-and-Fire for Adaptive Simultaneous Speech Translation](https://mlsanity.com/paper/2204.09595/) (2022)
- [Stream-level Latency Evaluation for Simultaneous Machine Translation](https://mlsanity.com/paper/2104.08817/) (2021)
- [Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning](https://mlsanity.com/paper/2002.04306/) (2020)
- [From Simultaneous to Streaming Machine Translation by Leveraging Streaming History](https://mlsanity.com/paper/2203.02459/) (2022)
- [End-to-End Speech-to-Text Translation: A Survey](https://mlsanity.com/paper/2312.01053/) (2023)
- [Towards the evaluation of automatic simultaneous speech translation from a communicative perspective](https://mlsanity.com/paper/2103.08364/) (2021)
- [Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems](https://mlsanity.com/paper/2110.15729/) (2021)
- [Continuous Rating as Reliable Human Evaluation of Simultaneous Speech Translation](https://mlsanity.com/paper/2203.02458/) (2022)
- [Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference](https://mlsanity.com/paper/2303.07914/) (2023)
- [Recent Advances in End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2406.00497/) (2024)
- [Impact of Encoding and Segmentation Strategies on End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2104.14470/) (2021)
- [Better Late Than Never: Meta-Evaluation of Latency Metrics for Simultaneous Speech-to-Text Translation](https://mlsanity.com/paper/2509.17349/) (2025)
- [Long-form Simultaneous Speech Translation: Thesis Proposal](https://mlsanity.com/paper/2310.11141/) (2023)
- [Visualization: the missing factor in Simultaneous Speech Translation](https://mlsanity.com/paper/2111.00514/) (2021)
- [A Discussion on Building Practical NLP Leaderboards: The Case of Machine Translation](https://mlsanity.com/paper/2106.06292/) (2021)
- [NAIST Simultaneous Speech Translation System for IWSLT 2024](https://mlsanity.com/paper/2407.00826/) (2024)
- [MFLA: Monotonic Finite Look-ahead Attention for Streaming Speech Recognition](https://mlsanity.com/paper/2506.03722/) (2025)
- [CA*: Addressing Evaluation Pitfalls in Computation-Aware Latency for Simultaneous Speech Translation](https://mlsanity.com/paper/2410.16011/) (2024)
- [TRADE: Transducer-Augmented Decoder for Speech LLM](https://mlsanity.com/paper/2606.08486/) (2026)
- [All In Good Time: Causality-Aware Framework for LLM-Based Simultaneous Speech-to-Speech Translation](https://mlsanity.com/paper/2609.30416/) (2026)
- [Simultaneous Translation between Sign Languages](https://mlsanity.com/paper/2609.35608/) (2026)

## Abstract (arXiv)

Simultaneous machine translation attempts to translate a source sentence before it is finished being spoken, with applications to translation of spoken language for live streaming and conversation. Since simultaneous systems trade quality to reduce latency, having an effective and interpretable latency metric is crucial. We introduce a variant of the recently proposed Average Lagging (AL) metric, which we call Differentiable Average Lagging (DAL). It distinguishes itself by being differentiable and internally consistent to its underlying mathematical model.