# Speech-to-speech Translation between Untranscribed Unknown Languages

arXiv: https://arxiv.org/abs/1910.00795 · HTML: https://mlsanity.com/paper/1910.00795/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Oct 2019.

## Built on by (most cited first, 27 of 27)

- [VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretation](https://mlsanity.com/paper/2101.00390/) (2021)
- [AudioPaLM: A Large Language Model That Can Speak and Listen](https://mlsanity.com/paper/2306.12925/) (2023)
- [Direct speech-to-speech translation with discrete units](https://mlsanity.com/paper/2107.05604/) (2021)
- [Textless Speech-to-Speech Translation on Real Data](https://mlsanity.com/paper/2112.08352/) (2021)
- [Translatotron 2: High-quality direct speech-to-speech translation with voice preservation](https://mlsanity.com/paper/2107.08661/) (2021)
- [CVSS Corpus and Massively Multilingual Speech-to-Speech Translation](https://mlsanity.com/paper/2201.03713/) (2022)
- [UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units](https://mlsanity.com/paper/2212.08055/) (2022)
- [Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation](https://mlsanity.com/paper/2204.02967/) (2022)
- [Recent Advances in Discrete Speech Tokens: A Review](https://mlsanity.com/paper/2502.06490/) (2025)
- [Self-supervised language learning from raw audio: Lessons from the Zero Resource Speech Challenge](https://mlsanity.com/paper/2210.15759/) (2022)
- [Speech-to-Speech Translation For A Real-world Unwritten Language](https://mlsanity.com/paper/2211.06474/) (2022)
- [Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation](https://mlsanity.com/paper/2203.13339/) (2022)
- [Textless Direct Speech-to-Speech Translation with Discrete Speech Representation](https://mlsanity.com/paper/2211.00115/) (2022)
- [PolyVoice: Language Models for Speech to Speech Translation](https://mlsanity.com/paper/2306.02982/) (2023)
- [AV2AV: Direct Audio-Visual Speech to Audio-Visual Speech Translation with Unified Audio-Visual Speech Representation](https://mlsanity.com/paper/2312.02512/) (2023)
- [Translatotron 3: Speech to Speech Translation with Monolingual Data](https://mlsanity.com/paper/2305.17547/) (2023)
- [Joint Pre-Training with Speech and Bilingual Text for Direct Speech to Speech Translation](https://mlsanity.com/paper/2210.17027/) (2022)
- [DASpeech: Directed Acyclic Transformer for Fast and High-quality Speech-to-Speech Translation](https://mlsanity.com/paper/2310.07403/) (2023)
- [AV-TranSpeech: Audio-Visual Robust Speech-to-Speech Translation](https://mlsanity.com/paper/2305.15403/) (2023)
- [When End-to-End is Overkill: Rethinking Cascaded Speech-to-Text Translation](https://mlsanity.com/paper/2502.00377/) (2025)
- [From Start to Finish: Latency Reduction Strategies for Incremental Speech Synthesis in Simultaneous Speech-to-Speech Translation](https://mlsanity.com/paper/2110.08214/) (2021)
- [Unsupervised Word Segmentation from Discrete Speech Units in Low-Resource Settings](https://mlsanity.com/paper/2106.04298/) (2021)
- [LLM-Ref: Enhancing Reference Handling in Technical Writing with Large Language Models](https://mlsanity.com/paper/2411.00294/) (2024)
- [DiffS2UT: A Semantic Preserving Diffusion Model for Textless Direct Speech-to-Speech Translation](https://mlsanity.com/paper/2310.17570/) (2023)
- [SimulTron: On-Device Simultaneous Speech to Speech Translation](https://mlsanity.com/paper/2406.02133/) (2024)
- [Leveraging Unit Language Guidance to Advance Speech Modeling in Textless Speech-to-Speech Translation](https://mlsanity.com/paper/2505.15333/) (2025)
- [DiffNorm: Self-Supervised Normalization for Non-autoregressive Speech-to-speech Translation](https://mlsanity.com/paper/2405.13274/) (2024)

## Abstract (arXiv)

In this paper, we explore a method for training speech-to-speech translation tasks without any transcription or linguistic supervision. Our proposed method consists of two steps: First, we train and generate discrete representation with unsupervised term discovery with a discrete quantized autoencoder. Second, we train a sequence-to-sequence model that directly maps the source language speech to the target language's discrete representation. Our proposed method can directly generate target speech without any auxiliary or pre-training steps with a source or target transcription. To the best of our knowledge, this is the first work that performed pure speech-to-speech translation between untranscribed unknown languages.