mlsanityData to 7 Oct 2026

Hierarchical Diffusion Models for Singing Voice Neural Vocoder

arXiv 2210.07508 ↗First posted 14 Oct 2022cs.SD cs.LG eess.AScited by 10

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent progress in deep generative models has improved the quality of neural vocoders in speech domain. However, generating a high-quality singing voice remains challenging due to a wider variety of musical expressions in pitch, loudness, and pronunciations. In this work, we propose a hierarchical diffusion model for singing voice neural vocoders. The proposed method consists of multiple diffusion models operating in different sampling rates; the model at the lowest sampling rate focuses on generating accurate low-frequency components such as pitch, and other models progressively generate the waveform at higher sampling rates on the basis of the data at the lower sampling rate and acoustic features. Experimental results show that the proposed method produces high-quality singing voices for multiple singers, outperforming state-of-the-art neural vocoders with a similar range of computational costs.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 28 of its 35 references that are in the map. First posted on arXiv 14 Oct 2022.

Builds on

28 of 35 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022SpecGrad: Diffusion Probabilistic Model based Neural Vocoder with Adaptive Noise Spectral Shaping31 Mar
  2. 2022Conditional Diffusion Probabilistic Model for Speech Enhancement10 Feb
  3. 2021PriorGrad: Improving Conditional Denoising Diffusion Models with Data-Dependent Adaptive Prior11 Jun
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  5. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  8. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  9. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  10. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  11. 2019Parallel WaveGAN: A fast waveform generation model based on generative adversarial networks with multi-resolution spectrogramcited by 337
  12. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 308
12 more
  1. 2018Adversarial Audio Synthesiscited by 222
  2. 2018Efficient Neural Audio Synthesiscited by 218
  3. 2019AUTOVC: Zero-Shot Voice Style Transfer with Only Autoencoder Losscited by 189
  4. 2016SampleRNN: An Unconditional End-to-End Neural Audio Generation Modelcited by 110
  5. 2018Generating High Fidelity Images with Subscale Pixel Networks and Multidimensional Upscalingcited by 74
  6. 2021StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversioncited by 57
  7. 2019WaveFlow: A Compact Flow-based Model for Raw Audiocited by 51
  8. 2021VoiceFixer: Toward General Speech Restoration with Neural Vocodercited by 50
  9. 2020HiFi-GAN: High-Fidelity Denoising and Dereverberation Based on Speech Deep Features in Adversarial Networkscited by 40
  10. 2020NHSS: A Speech and Singing Parallel Databasecited by 16
  11. 2019Parametric Resynthesis with neural vocoderscited by 8
  12. 2022Leveraging Symmetrical Convolutional Transformer Networks for Speech to Singing Voice Style Transfercited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 6; 2024: 2; 2025: 0; 2026: 1.

10 papers.

  1. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusioncited by 22
  2. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  3. 2024FreGrad: Lightweight and Fast Frequency-aware Diffusion Vocodercited by 11
  4. 2023Reconstruction of Sound Field through Diffusion Modelscited by 8
  5. 2023BigVSAN: Enhancing GAN-based Neural Vocoders with Slicing Adversarial Networkcited by 6
  6. 2022Robust One-Shot Singing Voice Conversioncited by 5
  7. 2024PeriodGrad: Towards Pitch-Controllable Neural Vocoder Based on a Diffusion Probabilistic Modelcited by 3
  8. 2023Iteratively Improving Speech Recognition and Voice Conversioncited by 1
  9. 2023Enhancing Semantic Communication with Deep Generative Models -- An ICASSP Special Session Overviewcited by 1
  10. 2026WaveNeXt 2: ConvNeXt-Based Fast Neural Vocoders With Residual Denoising and Sub-Modeling for GAN and Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023HiddenSinger: High-Quality Singing Voice Synthesis via Neural Audio Codec and Latent Diffusion Models
  2. 2025Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model
  3. 2025Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures
  4. 2025SmoothSinger: A Conditional Diffusion Model for Singing Voice Synthesis with Multi-Resolution Architecture
  5. 2026Adapting Diffusion-Based Music Synthesis to Speech and Singing Voice Conversion
  6. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanism
  7. 2022Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GAN
  8. 2024RDSinger: Reference-based Diffusion Network for Singing Voice Synthesis