mlsanityData to 7 Oct 2026

Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition

arXiv 2203.15796 ↗First posted 29 Mar 2022eess.AS cs.AIcited by 14

Not on the map. Why?

Abstract (arXiv)

An unsupervised text-to-speech synthesis (TTS) system learns to generate speech waveforms corresponding to any written sentence in a language by observing: 1) a collection of untranscribed speech waveforms in that language; 2) a collection of texts written in that language without access to any transcribed speech. Developing such a system can significantly improve the availability of speech technology to languages without a large amount of parallel speech and text data. This paper proposes an unsupervised TTS system based on an alignment module that outputs pseudo-text and another synthesis module that uses pseudo-text for training and real text for inference. Our unsupervised system can achieve comparable performance to the supervised system in seven languages with about 10-20 hours of speech each. A careful study on the effect of text units and vocoders has also been conducted to better understand what factors may affect unsupervised TTS performance. The samples generated by our models can be found at https://cactuswiththoughts.github.io/UnsupTTS-Demo, and our code can be found at https://github.com/lwang114/UnsupTTS.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 29 Mar 2022.

Builds on

21 of 32 references.

References · 21

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  3. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  6. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 462
  7. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  8. 2021Speech Resynthesis from Discrete Disentangled Self-Supervised Representationscited by 243
  9. 2021Unsupervised Speech Recognitioncited by 147
  10. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  11. 2017Deep Voice 3: Scaling Text-to-Speech with Convolutional Sequence Learningcited by 67
  12. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
9 more
  1. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  2. 2019CSS10: A Collection of Single Speaker Speech Datasets for 10 Languagescited by 54
  3. 2020Text-Free Image-to-Speech Synthesis Using Learned Segmental Unitscited by 50
  4. 2020LRSpeech: Extremely Low-Resource Speech Synthesis and Recognitioncited by 43
  5. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  6. 2018Unsupervised Speech Recognition via Segmental Empirical Output Distribution Matchingcited by 20
  7. 2019Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learningcited by 16
  8. 2020Unsupervised Learning For Sequence-to-sequence Text-to-speech For Low-resource Languagescited by 16
  9. 2020Show and Speak: Directly Synthesize Spoken Description of Imagescited by 1

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 6; 2024: 5; 2025: 1.

14 papers.

Sort:

  1. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  2. 2023Learning to Speak from Text: Zero-Shot Multilingual Text-to-Speech with Unsupervised Text Pretrainingcited by 10
  3. 2024Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognitioncited by 9
  4. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  5. 2023Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speechcited by 6
  6. 2023Improving Cascaded Unsupervised Speech Translation with Denoising Back-translationcited by 5
  7. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  8. 2024Improving Accented Speech Recognition using Data Augmentation based on Unsupervised Text-to-Speech Synthesiscited by 5
  9. 2023A Theory of Unsupervised Speech Recognitioncited by 4
  10. 2024A multilingual training strategy for low resource Text to Speechcited by 3
  11. 2025Towards Unsupervised Speech Recognition at the Syllable-Levelcited by 2
  12. 2023Iteratively Improving Speech Recognition and Voice Conversioncited by 1
2 more
  1. 2024Towards Unsupervised Speech Recognition Without Pronunciation Modelscited by 1
  2. 2024Maximizing Data Efficiency for Cross-Lingual TTS Adaptation by Self-Supervised Representation Mixing and Embedding Initializationcited by 0