mlsanityData to 14 Oct 2026

Controllable Generation of Artificial Speaker Embeddings through Discovery of Principal Directions

arXiv 2310.17502 ↗First posted 26 Oct 2023cs.SD cs.LG eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

Customizing voice and speaking style in a speech synthesis system with intuitive and fine-grained controls is challenging, given that little data with appropriate labels is available. Furthermore, editing an existing human's voice also comes with ethical concerns. In this paper, we propose a method to generate artificial speaker embeddings that cannot be linked to a real human while offering intuitive and fine-grained control over the voice and speaking style of the embeddings, without requiring any labels for speaker or style. The artificial and controllable embeddings can be fed to a speech synthesis system, conditioned on embeddings of real humans during training, without sacrificing privacy during inference.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 36 of its 54 references that are in the map. First posted on arXiv 26 Oct 2023.

Builds on

36 of 54 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017Improved Training of Wasserstein GANscited by 2,235
  3. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 717
  6. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 649
  7. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  8. 2020GANSpace: Discovering Interpretable GAN Controlscited by 614
  9. 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 476
  10. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 406
  11. 2018Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesiscited by 282
  12. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
24 more
  1. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  2. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  3. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  4. 2021Emotional Voice Conversion: Theory, Databases and ESDcited by 115
  5. 2020AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselinescited by 111
  6. 2018Neural Voice Cloning with a Few Samplescited by 100
  7. 2018Hierarchical Generative Modeling for Controllable Speech Synthesiscited by 93
  8. 2019Zero-Shot Multi-Speaker Text-To-Speech with State-of-the-art Neural Speaker Embeddingscited by 73
  9. 2021Hi-Fi Multi-Speaker English TTS Datasetcited by 60
  10. 2019CSS10: A Collection of Single Speaker Speech Datasets for 10 Languagescited by 54
  11. 2021PortaSpeech: Portable and High-Quality Generative Text-to-Speechcited by 47
  12. 2019Speaker Anonymization Using X-vector and Neural Waveform Modelscited by 41
  13. 2022AdaSpeech 4: Adaptive Text to Speech in Zero-Shot Scenarioscited by 41
  14. 2021Investigating on Incorporating Pretrained and Learnable Speaker Representations for Multi-Speaker Multi-Style Text-to-Speechcited by 37
  15. 2020Attentron: Few-Shot Text-to-Speech Utilizing Attention-Based Variable-Length Embeddingcited by 27
  16. 2022Anonymizing Speech with Generative Adversarial Networks to Preserve Speaker Privacycited by 21
  17. 2022Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Featurescited by 20
  18. 2021DelightfulTTS: The Microsoft Speech Synthesis System for Blizzard Challenge 2021cited by 19
  19. 2018Fitting New Speakers Based on a Short Untranscribed Samplecited by 18
  20. 2022Avocodo: Generative Adversarial Network for Artifact-free Vocodercited by 18
  21. 2021HUI-Audio-Corpus-German: A high quality TTS datasetcited by 15
  22. 2022Low-Resource Multilingual and Zero-Shot Multispeaker TTScited by 15
  23. 2021Adapting TTS models For New Speakers using Transfer Learningcited by 12
  24. 2022VoiceMe: Personalized voice generation in TTScited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026LISE : Listenable Interpretable Speaker Embeddingscited by 0