What does it take to get state of the art in simultaneous speech-to-speech translation?
Not on the map. Why?
Abstract (arXiv)
This paper presents an in-depth analysis of the latency characteristics observed in simultaneous speech-to-speech model's performance, particularly focusing on hallucination-induced latency spikes. By systematically experimenting with various input parameters and conditions, we propose methods to minimize latency spikes and improve overall performance. The findings suggest that a combination of careful input management and strategic parameter adjustments can significantly enhance speech-to-speech model's latency behavior.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 14 of its 25 references that are in the map. First posted on arXiv 2 Sep 2024.
Builds on
14 of 25 references.
References · 14
- 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 774
- 2015Listen, Attend and Spellcited by 542
- 2017Six Challenges for Neural Machine Translationcited by 334
- 2017State-of-the-art Speech Recognition With Sequence-to-Sequence Modelscited by 196
- 2022Paraformer: Fast and Accurate Parallel Transformer for Non-autoregressive End-to-End Speech Recognitioncited by 139
- 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 103
- 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
- 2016Can neural machine translation do simultaneous translation?cited by 85
- 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 80
- 2020Efficient Wait-k Models for Simultaneous Machine Translationcited by 59
- 2022Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translationcited by 56
- 2020Simultaneous Translation Policies: From Fixed to Adaptivecited by 42
Built on by
Citing papers per year: 2026: 1.
1 paper.