# Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation

arXiv: https://arxiv.org/abs/2305.07455 · HTML: https://mlsanity.com/paper/2305.07455/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 12 May 2023.

## Builds on (most cited first, 30 of 31)

- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Unsupervised Cross-lingual Representation Learning for Speech Recognition](https://mlsanity.com/paper/2006.13979/) (2020)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020)
- [Unsupervised Speech Recognition](https://mlsanity.com/paper/2105.11084/) (2021)
- [Sequence-to-Sequence Models Can Directly Translate Foreign Speech](https://mlsanity.com/paper/1703.08581/) (2017)
- [Direct speech-to-speech translation with a sequence-to-sequence model](https://mlsanity.com/paper/1904.06037/) (2019)
- [Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/1612.01744/) (2016)
- [CVSS Corpus and Massively Multilingual Speech-to-Speech Translation](https://mlsanity.com/paper/2201.03713/) (2022)
- [Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?](https://mlsanity.com/paper/2106.01045/) (2021)
- [Towards End-to-end Unsupervised Speech Recognition](https://mlsanity.com/paper/2204.02492/) (2022)
- [Almost Unsupervised Text to Speech and Automatic Speech Recognition](https://mlsanity.com/paper/1905.06791/) (2019)
- [Large-Scale Self- and Semi-Supervised Learning for Speech Translation](https://mlsanity.com/paper/2104.06678/) (2021)
- [Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spaces](https://mlsanity.com/paper/1805.07467/) (2018)
- [Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis](https://mlsanity.com/paper/1808.10128/) (2018)
- [Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddings](https://mlsanity.com/paper/1804.00316/) (2018)
- [ESPnet2-TTS: Extending the Edge of TTS Research](https://mlsanity.com/paper/2110.07840/) (2021)
- [Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition](https://mlsanity.com/paper/2203.15796/) (2022)
- [Robust Neural Machine Translation for Clean and Noisy Speech Transcripts](https://mlsanity.com/paper/1910.10238/) (2019)
- [Simple and Effective Unsupervised Speech Synthesis](https://mlsanity.com/paper/2204.02524/) (2022)
- [Simple and Effective Unsupervised Speech Translation](https://mlsanity.com/paper/2210.10191/) (2022)
- [Analyzing the Robustness of Unsupervised Speech Recognition](https://mlsanity.com/paper/2110.03509/) (2021)

## Built on by (most cited first, 5 of 5)

- [Direct Speech-to-Speech Neural Machine Translation: A Survey](https://mlsanity.com/paper/2411.14453/) (2024)
- [Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matching](https://mlsanity.com/paper/2310.02382/) (2023)
- [Textless Speech-to-Speech Translation With Limited Parallel Data](https://mlsanity.com/paper/2305.15405/) (2023)
- [SimulTron: On-Device Simultaneous Speech to Speech Translation](https://mlsanity.com/paper/2406.02133/) (2024)
- [Speech-to-Speech Translation Pipelines for Conversations in Low-Resource Languages](https://mlsanity.com/paper/2506.01406/) (2025)

## Abstract (arXiv)

Most of the speech translation models heavily rely on parallel data, which is hard to collect especially for low-resource languages. To tackle this issue, we propose to build a cascaded speech translation system without leveraging any kind of paired data. We use fully unpaired data to train our unsupervised systems and evaluate our results on CoVoST 2 and CVSS. The results show that our work is comparable with some other early supervised methods in some language pairs. While cascaded systems always suffer from severe error propagation problems, we proposed denoising back-translation (DBT), a novel approach to building robust unsupervised neural machine translation (UNMT). DBT successfully increases the BLEU score by 0.7--0.9 in all three translation directions. Moreover, we simplified the pipeline of our cascaded system to reduce inference latency and conducted a comprehensive analysis of every part of our work. We also demonstrate our unsupervised speech translation results on the established website.