Simultaneous Speech-to-Speech Translation System with Neural Incremental ASR, MT, and TTS
Not on the map. Why?
Abstract (arXiv)
This paper presents a newly developed, simultaneous neural speech-to-speech translation system and its evaluation. The system consists of three fully-incremental neural processing modules for automatic speech recognition (ASR), machine translation (MT), and text-to-speech synthesis (TTS). We investigated its overall latency in the system's Ear-Voice Span and speaking latency along with module-level performance.
Key facts. Cited by 12 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Nov 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 5; 2022: 1; 2023: 1; 2024: 2; 2025: 1; 2026: 2.
12 papers.
- 2024A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translationcited by 10
- 2021Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attentioncited by 9
- 2021Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Inputcited by 7
- 2021SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memorycited by 7
- 2021It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Datacited by 7
- 2025Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systemscited by 4
- 2021Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Networkcited by 2
- 2022Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representationscited by 1
- 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learningcited by 1
- 2024SimulTron: On-Device Simultaneous Speech to Speech Translationcited by 1
- 2026A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translationcited by 1
- 2026End-to-End Simultaneous Dysarthric Speech Reconstruction with Frame-Level Adaptor and Multiple Wait-k Knowledge Distillationcited by 0
- 2026A Practical Evaluation Method for Long-Form Simultaneous Speech-to-Speech Translation13 Jun 2026
- 2026End-to-End Simultaneous Dysarthric Speech Reconstruction with Frame-Level Adaptor and Multiple Wait-k Knowledge Distillation2 Mar 2026
- 2025Simulstream: Open-Source Toolkit for Evaluation and Demonstration of Streaming Speech-to-Text Translation Systems19 Dec 2025
- 2024A Non-autoregressive Generation Framework for End-to-End Simultaneous Speech-to-Speech Translation11 Jun 2024
- 2024SimulTron: On-Device Simultaneous Speech to Speech Translation4 Jun 2024
- 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learning31 Aug 2023
- 2022Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations27 Oct 2022
- 2021Direct Simultaneous Speech-to-Speech Translation with Variational Monotonic Multihead Attention15 Oct 2021
- 2021It is Not as Good as You Think! Evaluating Simultaneous Machine Translation on Interpretation Data11 Oct 2021
- 2021Low-Latency Incremental Text-to-Speech Synthesis with Distilled Context Prediction Network22 Sep 2021
- 2021SimulLR: Simultaneous Lip Reading Transducer with Attention-Guided Adaptive Memory31 Aug 2021
- 2021Alternate Endings: Improving Prosody for Incremental Neural TTS with Predicted Future Text Input19 Feb 2021