# Lost in Interpreting: Speech Translation from Source or Interpreter?

arXiv: https://arxiv.org/abs/2106.09343 · HTML: https://mlsanity.com/paper/2106.09343/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 17 Jun 2021.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019)
- [BLEU is Not Suitable for the Evaluation of Text Simplification](https://mlsanity.com/paper/1810.05995/) (2018)
- [Controlling Output Length in Neural Encoder-Decoders](https://mlsanity.com/paper/1609.09552/) (2016)
- [Positional Encoding to Control Output Sequence Length](https://mlsanity.com/paper/1904.07418/) (2019)
- [Low-Latency Neural Speech Translation](https://mlsanity.com/paper/1808.00491/) (2018)
- [Re-Translation Strategies For Long Form, Simultaneous, Spoken Language Translation](https://mlsanity.com/paper/1912.03393/) (2019)
- [Super-Human Performance in Online Low-latency Recognition of Conversational Speech](https://mlsanity.com/paper/2010.03449/) (2020)
- [Multilingual Neural Machine Translation for Zero-Resource Languages](https://mlsanity.com/paper/1909.07342/) (2019)
- [ELITR Non-Native Speech Translation at IWSLT 2020](https://mlsanity.com/paper/2006.03331/) (2020)

## Built on by (most cited first, 10 of 10)

- [Turning Whisper into Real-Time Transcription System](https://mlsanity.com/paper/2307.14743/) (2023)
- [Non-Parametric Domain Adaptation for End-to-End Speech Translation](https://mlsanity.com/paper/2205.11211/) (2022)
- [MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation](https://mlsanity.com/paper/2211.08633/) (2022)
- [Simultaneous Translation for Unsegmented Input: A Sliding Window Approach](https://mlsanity.com/paper/2210.09754/) (2022)
- [Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Pair](https://mlsanity.com/paper/2404.12299/) (2024)
- [NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpus](https://mlsanity.com/paper/2304.11766/) (2023)
- [Robustness of Multi-Source MT to Transcription Errors](https://mlsanity.com/paper/2305.16894/) (2023)
- [MockConf: A Student Interpretation Dataset: Analysis, Word- and Span-level Alignment and Baselines](https://mlsanity.com/paper/2506.04848/) (2025)
- [Rethinking the Reasonability of the Test Set for Simultaneous Machine Translation](https://mlsanity.com/paper/2303.00969/) (2023)
- [Whispy: Adapting STT Whisper Models to Real-Time Environments](https://mlsanity.com/paper/2405.03484/) (2024)

## Abstract (arXiv)

Interpreters facilitate multi-lingual meetings but the affordable set of languages is often smaller than what is needed. Automatic simultaneous speech translation can extend the set of provided languages. We investigate if such an automatic system should rather follow the original speaker, or an interpreter to achieve better translation quality at the cost of increased delay. To answer the question, we release Europarl Simultaneous Interpreting Corpus (ESIC), 10 hours of recordings and transcripts of European Parliament speeches in English, with simultaneous interpreting into Czech and German. We evaluate quality and latency of speaker-based and interpreter-based spoken translation systems from English to Czech. We study the differences in implicit simplification and summarization of the human interpreter compared to a machine translation system trained to shorten the output to some extent. Finally, we perform human evaluation to measure information loss of each of these approaches.