mlsanityData to 14 Oct 2026

SpecDiff-GAN: A Spectrally-Shaped Noise Diffusion GAN for Speech and Music Synthesis

arXiv 2402.01753 ↗First posted 30 Jan 2024cs.SD cs.LG eess.AScited by 3

Not on the map. Why?

Abstract (arXiv)

Generative adversarial network (GAN) models can synthesize highquality audio signals while ensuring fast sample generation. However, they are difficult to train and are prone to several issues including mode collapse and divergence. In this paper, we introduce SpecDiff-GAN, a neural vocoder based on HiFi-GAN, which was initially devised for speech synthesis from mel spectrogram. In our model, the training stability is enhanced by means of a forward diffusion process which consists in injecting noise from a Gaussian distribution to both real and fake samples before inputting them to the discriminator. We further improve the model by exploiting a spectrally-shaped noise distribution with the aim to make the discriminator's task more challenging. We then show the merits of our proposed model for speech and music synthesis on several datasets. Our experiments confirm that our model compares favorably in audio quality and efficiency compared to several baselines.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 21 of its 32 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

21 of 32 references.

References · 21

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  5. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
  6. 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  8. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
  9. 2022BigVGAN: A Universal Neural Vocoder with Large-Scale Trainingcited by 251
  10. 2022Diffusion-GAN: Training GANs with Diffusioncited by 205
  11. 2019Direct speech-to-speech translation with a sequence-to-sequence modelcited by 118
  12. 2019GANSynth: Adversarial Neural Audio Synthesiscited by 116
9 more
  1. 2020An Overview of Voice Conversion and its Challenges: From Statistical Modeling to Deep Learningcited by 112
  2. 2020Neural Networks Fail to Learn Periodic Functions and How to Fix Itcited by 100
  3. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Priorcited by 80
  4. 2021UnivNet: A Neural Vocoder with Multi-Resolution Spectrogram Discriminators for High-Fidelity Waveform Generationcited by 74
  5. 2020A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directionscited by 56
  6. 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shapingcited by 37
  7. 2021WARP-Q: Quality Prediction For Generative Neural Speech Codecscited by 12
  8. 2022HiFi-WaveGAN: Generative Adversarial Network with Auxiliary Spectrogram-Phase Loss for High-Fidelity Singing Voice Generationcited by 2
  9. 2023Enhancing Unsupervised Speech Recognition with Diffusion GANscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Integrating Text-to-Music Models with Language Models: Composing Long Structured Music Piecescited by 0
  2. 2025GLA-Grad++: An Improved Griffin-Lim Guided Diffusion Model for Speech Synthesiscited by 0
  3. 2026WaveNeXt 2: ConvNeXt-Based Fast Neural Vocoders With Residual Denoising and Sub-Modeling for GAN and Diffusion Modelscited by 0