# Low-Latency Neural Speech Translation

arXiv: https://arxiv.org/abs/1808.00491 · HTML: https://mlsanity.com/paper/1808.00491/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Aug 2018.

## Built on by (most cited first, 30 of 37)

- [Face-Dubbing++: Lip-Synchronous, Voice Preserving Translation of Videos](https://mlsanity.com/paper/2206.04523/) (2022)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [BSTC: A Large-Scale Chinese-English Speech Translation Dataset](https://mlsanity.com/paper/2104.03575/) (2021)
- [An Empirical Study of End-to-end Simultaneous Speech Translation Decoding Strategies](https://mlsanity.com/paper/2103.03233/) (2021)
- [Learning Coupled Policies for Simultaneous Machine Translation using Imitation Learning](https://mlsanity.com/paper/2002.04306/) (2020)
- [Streaming Models for Joint Speech Recognition and Translation](https://mlsanity.com/paper/2101.09149/) (2021)
- [Lost in Interpreting: Speech Translation from Source or Interpreter?](https://mlsanity.com/paper/2106.09343/) (2021)
- [End-to-End Evaluation for Low-Latency Simultaneous Speech Translation](https://mlsanity.com/paper/2308.03415/) (2023)
- [From Simultaneous to Streaming Machine Translation by Leveraging Streaming History](https://mlsanity.com/paper/2203.02459/) (2022)
- [Simultaneous Machine Translation with Large Language Models](https://mlsanity.com/paper/2309.06706/) (2023)
- [Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation](https://mlsanity.com/paper/2210.13363/) (2022)
- [Incremental Blockwise Beam Search for Simultaneous Speech Translation with Controllable Quality-Latency Tradeoff](https://mlsanity.com/paper/2309.11379/) (2023)
- [Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation](https://mlsanity.com/paper/2102.11387/) (2021)
- [Continuous Rating as Reliable Human Evaluation of Simultaneous Speech Translation](https://mlsanity.com/paper/2203.02458/) (2022)
- [Token-Level Serialized Output Training for Joint Streaming ASR and ST Leveraging Textual Alignments](https://mlsanity.com/paper/2307.03354/) (2023)
- [Long-Form End-to-End Speech Translation via Latent Alignment Segmentation](https://mlsanity.com/paper/2309.11384/) (2023)
- [Recent Advances in End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2406.00497/) (2024)
- [Impact of Encoding and Segmentation Strategies on End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2104.14470/) (2021)
- [Towards Real-World Streaming Speech Translation for Code-Switched Speech](https://mlsanity.com/paper/2310.12648/) (2023)
- [Leveraging Timestamp Information for Serialized Joint Streaming Recognition and Translation](https://mlsanity.com/paper/2310.14806/) (2023)
- [Optimizing Latency for Online Video CaptioningUsing Audio-Visual Transformers](https://mlsanity.com/paper/2108.02147/) (2021)
- [Long-form Simultaneous Speech Translation: Thesis Proposal](https://mlsanity.com/paper/2310.11141/) (2023)
- [Visualization: the missing factor in Simultaneous Speech Translation](https://mlsanity.com/paper/2111.00514/) (2021)
- [Supervised Visual Attention for Simultaneous Multimodal Machine Translation](https://mlsanity.com/paper/2201.09324/) (2022)
- [Multilingual Simultaneous Speech Translation](https://mlsanity.com/paper/2203.14835/) (2022)
- [Robustness of Multi-Source MT to Transcription Errors](https://mlsanity.com/paper/2305.16894/) (2023)
- [MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation task](https://mlsanity.com/paper/2506.18828/) (2025)
- [MeetDot: Videoconferencing with Live Translation Captions](https://mlsanity.com/paper/2109.09577/) (2021)
- [Streaming Non-Autoregressive Model for Accent Conversion and Pronunciation Improvement](https://mlsanity.com/paper/2506.16580/) (2025)
- [Assessing Identity Leakage in Talking Face Generation: Metrics and Evaluation Framework](https://mlsanity.com/paper/2511.08613/) (2025)

## Abstract (arXiv)

Through the development of neural machine translation, the quality of machine translation systems has been improved significantly. By exploiting advancements in deep learning, systems are now able to better approximate the complex mapping from source sentences to target sentences. But with this ability, new challenges also arise. An example is the translation of partial sentences in low-latency speech translation. Since the model has only seen complete sentences in training, it will always try to generate a complete sentence, though the input may only be a partial sentence. We show that NMT systems can be adapted to scenarios where no task-specific training data is available. Furthermore, this is possible without losing performance on the original training data. We achieve this by creating artificial data and by using multi-task learning. After adaptation, we are able to reduce the number of corrections displayed during incremental output construction by 45%, without a decrease in translation quality.