mlsanityData to 7 Oct 2026

Generative Data Augmentation Challenge: Zero-Shot Speech Synthesis for Personalized Speech Enhancement

arXiv 2501.13372 ↗First posted 23 Jan 2025eess.AS cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

This paper presents a new challenge that calls for zero-shot text-to-speech (TTS) systems to augment speech data for the downstream task, personalized speech enhancement (PSE), as part of the Generative Data Augmentation workshop at ICASSP 2025. Collecting high-quality personalized data is challenging due to privacy concerns and technical difficulties in recording audio from the test scene. To address these issues, synthetic data generation using generative models has gained significant attention. In this challenge, participants are tasked first with building zero-shot TTS systems to augment personalized data. Subsequently, PSE systems are asked to be trained with this augmented personalized dataset. Through this challenge, we aim to investigate how the quality of augmented data generated by zero-shot TTS models affects PSE model performance. We also provide baseline experiments using open-source zero-shot TTS models to encourage participation and benchmark advancements. Our baseline code implementation and checkpoints are available online.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 29 of its 36 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

29 of 36 references.

References · 29

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
  4. 2019LibriTTS: A Corpus Derived from LibriSpeech for Text-to-Speechcited by 639
  5. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  6. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  7. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 400
  8. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  9. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 339
  10. 2023Effective Data Augmentation With Diffusion Modelscited by 282
  11. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  12. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
17 more
  1. 2021NISQA: A Deep CNN-Self-Attention Model for Multidimensional Speech Quality Prediction with Crowdsourced Datasetscited by 189
  2. 2021SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processingcited by 177
  3. 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 174
  4. 2021MetricGAN+: An Improved Version of MetricGAN for Speech Enhancementcited by 114
  5. 2024The T05 System for The VoiceMOS Challenge 2024: Transfer Learning from Deep Image Classifier to Naturalness MOS Prediction of High-Quality Synthetic Speechcited by 78
  6. 2023Better speech synthesis through scalingcited by 63
  7. 2022MANNER: Multi-view Attention Network for Noise Erasurecited by 28
  8. 2023The Singing Voice Conversion Challenge 2023cited by 25
  9. 2024The VoiceMOS Challenge 2024: Beyond Speech Quality Predictioncited by 17
  10. 2023Text is All You Need: Personalizing ASR Models using Controllable Speech Synthesiscited by 9
  11. 2023Improving Few-Shot Learning for Talking Face System with TTS Data Augmentationcited by 6
  12. 2021Efficient Personalized Speech Enhancement through Self-Supervised Learningcited by 5
  13. 2020Deep MOS Predictor for Synthetic Speech Using Cluster-Based Modelingcited by 4
  14. 2023Latent Filling: Latent Space Data Augmentation for Zero-shot Speech Synthesiscited by 4
  15. 2024Utilizing TTS Synthesized Data for Efficient Development of Keyword Spotting Modelcited by 4
  16. 2022The Potential of Neural Speech Synthesis-based Data Augmentation for Personalized Speech Enhancementcited by 2
  17. 2024Zero Shot Text to Speech Augmentation for Automatic Speech Recognition on Low-Resource Accented Speech Corporacited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Analyzing and Improving Speaker Similarity Assessment for Speech Synthesiscited by 3
  2. 2026Semantics-Aware Generative Latent Data Augmentation for Learning in Low-Resource Domainscited by 0
  3. 2026ZeSTA: Zero-Shot TTS Augmentation with Domain-Conditioned Training for Data-Efficient Personalized Speech Synthesiscited by 0