mlsanityData to 7 Oct 2026

Simple and Effective Unsupervised Speech Synthesis

arXiv 2204.02524 ↗First posted 6 Apr 2022cs.SD cs.CL eess.AScited by 12

Not on the map. Why?

Abstract (arXiv)

We introduce the first unsupervised speech synthesis system based on a simple, yet effective recipe. The framework leverages recent work in unsupervised speech recognition as well as existing neural-based speech synthesis. Using only unlabeled speech audio and unlabeled text as well as a lexicon, our method enables speech synthesis without the need for a human-labeled corpus. Experiments demonstrate the unsupervised system can synthesize speech similar to a supervised counterpart in terms of naturalness and intelligibility measured by human evaluation.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 27 of its 33 references that are in the map. First posted on arXiv 6 Apr 2022.

Builds on

27 of 33 references.

References · 27

  1. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  6. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  7. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  8. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  9. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  10. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  11. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  12. 2019MelGAN: Generative Adversarial Networks for Conditional Waveform Synthesiscited by 351
15 more
  1. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  2. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 280
  3. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 257
  4. 2021A Survey on Neural Speech Synthesiscited by 182
  5. 2021Unsupervised Speech Recognitioncited by 147
  6. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  7. 2019ESPnet-TTS: Unified, Reproducible, and Integratable Open Source End-to-End Text-to-Speech Toolkitcited by 60
  8. 2017Efficiently Trainable Text-to-Speech System Based on Deep Convolutional Networks with Guided Attentioncited by 57
  9. 2021PnG BERT: Augmented BERT on Phonemes and Graphemes for Neural TTScited by 52
  10. 2017Listening while Speaking: Speech Chain by Deep Learningcited by 46
  11. 2022Towards End-to-end Unsupervised Speech Recognitioncited by 44
  12. 2019Almost Unsupervised Text to Speech and Automatic Speech Recognitioncited by 41
  13. 2018Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesiscited by 30
  14. 2019Towards Unsupervised Speech Recognition and Synthesis with Quantized Speech Representation Learningcited by 16
  15. 2022ECAPA-TDNN for Multi-speaker Text-to-speech Synthesiscited by 5

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 6; 2024: 2; 2025: 1.

12 papers.

  1. 2022Simple and Effective Unsupervised Speech Translationcited by 12
  2. 2022Unsupervised TTS Acoustic Modeling for TTS with Conditional Disentangled Sequential VAEcited by 7
  3. 2023Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speechcited by 6
  4. 2023Improving Cascaded Unsupervised Speech Translation with Denoising Back-translationcited by 5
  5. 2023Unsupervised Speech Recognition with N-Skipgram and Positional Unigram Matchingcited by 5
  6. 2024Fake it to make it: Using synthetic data to remedy the data shortage in joint multimodal speech-and-gesture synthesiscited by 5
  7. 2023ParrotTTS: Text-to-Speech synthesis by exploiting self-supervised representationscited by 4
  8. 2022Learning the joint distribution of two sequences using little or no paired datacited by 3
  9. 2023Textless Speech-to-Speech Translation With Limited Parallel Datacited by 3
  10. 2024A multilingual training strategy for low resource Text to Speechcited by 3
  11. 2023Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audiocited by 2
  12. 2025Towards Unsupervised Speech Recognition at the Syllable-Levelcited by 2