mlsanityData to 7 Oct 2026

Learning Latent Representations for Speech Generation and Transformation

arXiv 1704.04222 ↗First posted 13 Apr 2017cs.CL cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

An ability to model a generative process and learn a latent representation for speech in an unsupervised fashion will be crucial to process vast quantities of unlabelled speech data. Recently, deep probabilistic generative models such as Variational Autoencoders (VAEs) have achieved tremendous success in modeling natural images. In this paper, we apply a convolutional VAE to model the generative process of natural speech. We derive latent space arithmetic operations to disentangle learned latent representations. We demonstrate the capability of our model to modify the phonetic content or the speaker identity for speech segments using the derived operations, without the need for parallel supervisory data.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Apr 2017.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 10; 2022: 11; 2023: 3; 2024: 2; 2025: 1.

27 papers.

Sort:

  1. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  2. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  3. 2021Generative Spoken Language Modeling from Raw Audiocited by 336
  4. 2022Self-Supervised Speech Representation Learning: A Reviewcited by 258
  5. 2023Deep networks for system identification: a Surveycited by 35
  6. 2021A Survey on Deep Reinforcement Learning for Audio-Based Applicationscited by 17
  7. 2025OmniAudio: Generating Spatial Audio from 360-Degree Videocited by 17
  8. 2021Emphasis control for parallel neural TTScited by 9
  9. 2022A Brief Overview of Unsupervised Neural Speech Representation Learningcited by 9
  10. 2022Gromov-Wasserstein Autoencoderscited by 7
  11. 2022Losses Can Be Blessings: Routing Self-Supervised Speech Representations Towards Efficient Multilingual and Multitask Speech Processingcited by 7
  12. 2021Pathological voice adaptation with autoencoder-based voice conversioncited by 5
15 more
  1. 2022Modeling speech recognition and synthesis simultaneously: Encoding and decoding lexical and sublexical semantic information into speech with no direct access to speech datacited by 5
  2. 2021Learning robust speech representation with an articulatory-regularized variational autoencodercited by 4
  3. 2021A learned conditional prior for the VAE acoustic space of a TTS systemcited by 4
  4. 2022A Sparsity-promoting Dictionary Model for Variational Autoencoderscited by 2
  5. 2022Learning and controlling the source-filter representation of speech with a variational autoencodercited by 2
  6. 2021An Attribute-Aligned Strategy for Learning Speech Representationcited by 1
  7. 2023Cross-Utterance Conditioned VAE for Speech Generationcited by 1
  8. 2024Interference Motion Removal for Doppler Radar Vital Sign Detection Using Variational Encoder-Decoder Neural Networkcited by 1
  9. 2024Towards the Next Frontier in Speech Representation Learning Using Disentanglementcited by 1
  10. 2021Towards Cross-Cultural Analysis using Music Information Dynamicscited by 0
  11. 2022Improved far-field speech recognition using Joint Variational Autoencodercited by 0
  12. 2022Learning Multivariate CDFs and Copulas using Tensor Factorizationcited by 0
  13. 2022Learning Invariant Representation and Risk Minimized for Unsupervised Accent Domain Adaptationcited by 0
  14. 2022Privacy-Utility Balanced Voice De-Identification Using Adversarial Examplescited by 0
  15. 2023An investigation of the reconstruction capacity of stacked convolutional autoencoders for log-mel-spectrogramscited by 0