# Average Token Delay: A Duration-aware Latency Metric for Simultaneous Translation

arXiv: https://arxiv.org/abs/2311.14353 · HTML: https://mlsanity.com/paper/2311.14353/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 24 Nov 2023.

## Builds on (most cited first, 10 of 10)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [WhisperX: Time-Accurate Speech Transcription of Long-Form Audio](https://mlsanity.com/paper/2303.00747/) (2023)
- [Word Alignment by Fine-tuning Embeddings on Parallel Corpora](https://mlsanity.com/paper/2101.08231/) (2021)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [SimulEval: An Evaluation Toolkit for Simultaneous Translation](https://mlsanity.com/paper/2007.16193/) (2020)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation](https://mlsanity.com/paper/2206.05807/) (2022)
- [Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selection](https://mlsanity.com/paper/2005.11185/) (2020)
- [Average Token Delay: A Latency Metric for Simultaneous Translation](https://mlsanity.com/paper/2211.13173/) (2022)
- [Stream-level Latency Evaluation for Simultaneous Machine Translation](https://mlsanity.com/paper/2104.08817/) (2021)

## Built on by (most cited first, 2 of 2)

- [NAIST Simultaneous Speech Translation System for IWSLT 2024](https://mlsanity.com/paper/2407.00826/) (2024)
- [Simultaneous Translation between Sign Languages](https://mlsanity.com/paper/2609.35608/) (2026)

## Abstract (arXiv)

Simultaneous translation is a task in which the translation begins before the end of an input speech segment. Its evaluation should be conducted based on latency in addition to quality, and for users, the smallest possible amount of latency is preferable. Most existing metrics measure latency based on the start timings of partial translations and ignore their duration. This means such metrics do not penalize the latency caused by long translation output, which delays the comprehension of users and subsequent translations. In this work, we propose a novel latency evaluation metric for simultaneous translation called Average Token Delay (ATD) that focuses on the duration of partial translations. We demonstrate its effectiveness through analyses simulating user-side latency based on Ear-Voice Span (EVS). In our experiment, ATD had the highest correlation with EVS among baseline latency metrics under most conditions.