mlsanityData to 14 Oct 2026

Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolution

arXiv 2210.15793 ↗First posted 27 Oct 2022eess.AS cs.SD eess.SPcited by 20

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recently, diffusion models (DMs) have been increasingly used in audio processing tasks, including speech super-resolution (SR), which aims to restore high-frequency content given low-resolution speech utterances. This is commonly achieved by conditioning the network of noise predictor with low-resolution audio. In this paper, we propose a novel sampling algorithm that communicates the information of the low-resolution audio via the reverse sampling process of DMs. The proposed method can be a drop-in replacement for the vanilla sampling process and can significantly improve the performance of the existing works. Moreover, by coupling the proposed sampling method with an unconditional DM, i.e., a DM with no auxiliary inputs to its noise predictor, we can generalize it to a wide range of SR setups. We also attain state-of-the-art results on the VCTK Multi-Speaker benchmark with this novel formulation.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 18 of its 23 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

18 of 23 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain31 Mar
  2. 2021A Study on Speech Enhancement Based on Diffusion Probabilistic Model25 Jul
  3. 2021NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling6 Apr
  4. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  6. 2021Variational Diffusion Modelscited by 1,153
  7. 2022Denoising Diffusion Restoration Modelscited by 963
  8. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  9. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 521
  10. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 508
  11. 2018WaveGlow: A Flow-based Generative Network for Speech Synthesiscited by 309
  12. 2022Neural Vocoder is All You Need for Speech Super-resolutioncited by 37
2 more
  1. 2022NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Ratescited by 36
  2. 2021WSRGlow: A Glow-based Waveform Generative Model for Audio Super-Resolutioncited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 8; 2024: 3; 2025: 6; 2026: 3.

20 papers.

Sort:

By area:

  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 73
  2. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  3. 2023Multi-Source Diffusion Models for Simultaneous Music Generation and Separationcited by 63
  4. 2023HierSpeech++: Bridging the Gap between Semantic and Acoustic Representation of Speech by Hierarchical Variational Inference for Zero-shot Speech Synthesiscited by 35
  5. 2023Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approachcited by 21
  6. 2025FLowHigh: Towards Efficient and High-Quality Audio Super-Resolution with Single-Step Flow Matchingcited by 18
  7. 2025Audio Super-Resolution with Latent Bridge Modelscited by 14
  8. 2025A2SB: Audio-to-Audio Schrodinger Bridgescited by 11
  9. 2023Reconstruction of Sound Field through Diffusion Modelscited by 8
  10. 2024Naturalistic Music Decoding from EEG Data via Latent Diffusion Modelscited by 8
  11. 2023Thompson Sampling with Diffusion Generative Priorcited by 7
  12. 2023mdctGAN: Taming transformer-based GAN for speech super-resolution with Modified DCT spectracited by 6
8 more
  1. 2023Zero-Shot Duet Singing Voices Separation with Diffusion Modelscited by 6
  2. 2025Inference-time Scaling for Diffusion-based Audio Super-resolutioncited by 3
  3. 2025UniverSR: Unified and Versatile Audio Super-Resolution via Vocoder-Free Flow Matchingcited by 3
  4. 2024MSECG: Incorporating Mamba for Robust and Efficient ECG Super-Resolutioncited by 2
  5. 2026LatentFlowSR: High-Fidelity Audio Super-Resolution via Noise-Robust Latent Flow Matchingcited by 1
  6. 2025Improving Inference-Time Optimisation for Vocal Effects Style Transfer with a Gaussian Priorcited by 0
  7. 2026FastWave: Optimized Diffusion Model for Audio Super-Resolutioncited by 0
  8. 2026P2Flow: Phoneme-aware Progressive Flow Matching for Extreme Speech Super-Resolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis
  2. 2025FlashSR: One-step Versatile Audio Super-resolution via Diffusion Distillation
  3. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation
  4. 2022Cold Diffusion for Speech Enhancement
  5. 2023CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model
  6. 2025On the Application of Diffusion Models for Simultaneous Denoising and Dereverberation
  7. 2024Diffusion Models for Audio Restoration
  8. 2026SSNAPS: Audio-Visual Separation of Speech and Background Noise with Diffusion Inverse Sampling