mlsanityData to 7 Oct 2026

Improving Cascaded Unsupervised Speech Translation with Denoising Back-translation

arXiv 2305.07455 ↗First posted 12 May 2023cs.CL cs.SD eess.AScited by 5

Not on the map. Why?

Abstract (arXiv)

Most of the speech translation models heavily rely on parallel data, which is hard to collect especially for low-resource languages. To tackle this issue, we propose to build a cascaded speech translation system without leveraging any kind of paired data. We use fully unpaired data to train our unsupervised systems and evaluate our results on CoVoST 2 and CVSS. The results show that our work is comparable with some other early supervised methods in some language pairs. While cascaded systems always suffer from severe error propagation problems, we proposed denoising back-translation (DBT), a novel approach to building robust unsupervised neural machine translation (UNMT). DBT successfully increases the BLEU score by 0.7--0.9 in all three translation directions. Moreover, we simplified the pipeline of our cascaded system to reduce inference latency and conducted a comprehensive analysis of every part of our work. We also demonstrate our unsupervised speech translation results on the established website.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 31 of its 44 references that are in the map. First posted on arXiv 12 May 2023.

Builds on

31 of 44 references.

References · 31

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  8. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  9. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
  10. 2017Synthetic and Natural Noise Both Break Neural Machine Translationcited by 304
  11. 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
  12. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
19 more
  1. 2021Unsupervised Speech Recognitioncited by 147
  2. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  3. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  4. 2016Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translationcited by 113
  5. 2022CVSS Corpus and Massively Multilingual Speech-to-Speech Translationcited by 71
  6. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  7. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  8. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  9. 2021Large-Scale Self- and Semi-Supervised Learning for Speech Translationcited by 36
  10. 2018Unsupervised Cross-Modal Alignment of Speech and Text Embedding Spacescited by 34
  11. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  12. 2018Completely Unsupervised Phoneme Recognition by Adversarially Learning Mapping Relationships from Audio Embeddingscited by 22
  13. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  14. 2022Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognitioncited by 14
  15. 2019Robust Neural Machine Translation for Clean and Noisy Speech Transcriptscited by 13
  16. 2022Simple and Effective Unsupervised Speech Synthesiscited by 12
  17. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  18. 2021Analyzing the Robustness of Unsupervised Speech Recognitioncited by 11
  19. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  2. 2024Direct Speech-to-Speech Neural Machine Translation: A Surveycited by 5
  3. 2023Textless Speech-to-Speech Translation With Limited Parallel Datacited by 3
  4. 2024SimulTron: On-Device Simultaneous Speech to Speech Translationcited by 1
  5. 2025Speech-to-Speech Translation Pipelines for Conversations in Low-Resource Languagescited by 0