mlsanityData to 7 Oct 2026

Solving Audio Inverse Problems with a Diffusion Model

arXiv 2210.15228 ↗First posted 27 Oct 2022eess.AS cs.SDcited by 62

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

This paper presents CQT-Diff, a data-driven generative audio model that can, once trained, be used for solving various different audio inverse problems in a problem-agnostic setting. CQT-Diff is a neural diffusion model with an architecture that is carefully constructed to exploit pitch-equivariant symmetries in music. This is achieved by preconditioning the model with an invertible Constant-Q Transform (CQT), whose logarithmically-spaced frequency axis represents pitch equivariance as translation equivariance. The proposed method is evaluated with objective and subjective metrics in three different and varied tasks: audio bandwidth extension, inpainting, and declipping. The results show that CQT-Diff outperforms the compared baselines and ablations in audio bandwidth extension and, without retraining, delivers competitive performance against modern baselines in audio inpainting and declipping. This work represents the first diffusion-based general framework for solving inverse problems in audio processing.

Key facts. Cited by 62 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 30 of its 36 references that are in the map. First posted on arXiv 27 Oct 2022.

Builds on

30 of 36 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Models11 Aug
  2. 2022Realistic Gramophone Noise Synthesis using a Diffusion Model13 Jun
  3. 2022Multi-instrument Music Synthesis with Spectrogram Diffusion11 Jun
  4. 2022Guided-TTS 2: A Diffusion Model for High-quality Adaptive Text-to-Speech with Untranscribed Data30 May
  5. 2021CRASH: Raw Audio Score-based Generative Modeling for Controllable High-resolution Drum Sound Synthesis14 Jun
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesis21 Sep

Other references · 24

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  6. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,286
  7. 2022Video Diffusion Modelscited by 2,280
  8. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  9. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  10. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  11. 2022Denoising Diffusion Restoration Modelscited by 951
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
12 more
  1. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  2. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  3. 2022It's Raw! Audio Generation with State-Space Modelscited by 171
  4. 2022Universal Speech Enhancement with Score-based Diffusioncited by 75
  5. 2021Sequence-to-Sequence Piano Transcription with Transformerscited by 48
  6. 2022NU-Wave 2: A General Neural Audio Upsampling Model for Various Sampling Ratescited by 36
  7. 2018TimbreTron: A WaveNet(CycleGAN(CQT(Audio))) Pipeline for Musical Timbre Transfercited by 32
  8. 2020GACELA -- A generative adversarial context encoder for long audio inpaintingcited by 15
  9. 2021Catch-A-Waveform: Learning to Generate Audio from a Single Short Examplecited by 15
  10. 2022BEHM-GAN: Bandwidth Extension of Historical Music using Generative Adversarial Networkscited by 13
  11. 2020On Filter Generalization for Music Bandwidth Extension Using Deep Neural Networkscited by 10
  12. 2015Sparsity and cosparsity for audio declipping: a flexible non-convex approachcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 11; 2024: 19; 2025: 18; 2026: 14.

62 papers.

Sort:

By area:

  1. 2024Fast Timing-Conditioned Latent Audio Diffusioncited by 178
  2. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  3. 2024TFG: Unified Training-Free Guidance for Diffusion Modelscited by 103
  4. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  5. 2024Diffusion Models for Audio Restorationcited by 32
  6. 2023Controllable Music Production with Diffusion Models and Guidance Gradientscited by 29
  7. 2023Blind Audio Bandwidth Extension: A Diffusion-Based Zero-Shot Approachcited by 21
  8. 2023Diffusion models for audio semantic communicationcited by 17
  9. 2025FlowDec: A flow-based full-band general audio codec with high perceptual qualitycited by 15
  10. 2023DiffVoice: Text-to-Speech with Latent Diffusioncited by 13
  11. 2025Audio Super-Resolution with Latent Bridge Modelscited by 13
  12. 2024A Diffusion-Based Generative Equalizer for Music Restorationcited by 12
38 more
  1. 2024Unsupervised Blind Joint Dereverberation and Room Acoustics Estimation with Diffusion Modelscited by 11
  2. 2025A2SB: Audio-to-Audio Schrodinger Bridgescited by 11
  3. 2023Investigating the Design Space of Diffusion Models for Speech Enhancementcited by 10
  4. 2023Diffusion Posterior Sampling for Informed Single-Channel Dereverberationcited by 9
  5. 2024BUDDy: Single-Channel Blind Unsupervised Dereverberation with Diffusion Modelscited by 9
  6. 2024SoundCTM: Unifying Score-based and Consistency Models for Full-band Text-to-Sound Generationcited by 9
  7. 2024Latent Diffusion Bridges for Unsupervised Musical Audio Timbre Transfercited by 9
  8. 2023Reconstruction of Sound Field through Diffusion Modelscited by 8
  9. 2024Apollo: Band-sequence Modeling for High-Quality Audio Restorationcited by 8
  10. 2023UnDiff: Unsupervised Voice Restoration with Unconditional Diffusion Modelcited by 7
  11. 2025Inverse Problem Sampling in Latent Space Using Sequential Monte Carlocited by 7
  12. 2025ArrayDPS: Unsupervised Blind Speech Separation with a Diffusion Priorcited by 7
  13. 2023Zero-Shot Duet Singing Voices Separation with Diffusion Modelscited by 6
  14. 2024A Variance-Preserving Interpolation Approach for Diffusion Models with Applications to Single Channel Speech Enhancement and Recognitioncited by 5
  15. 2025Unsupervised Estimation of Nonlinear Audio Effects: Comparing Diffusion-Based and Adversarial approachescited by 4
  16. 2023Timbre-Trap: A Low-Resource Framework for Instrument-Agnostic Music Transcriptioncited by 3
  17. 2024Training-Free Multi-User Generative Semantic Communications via Null-Space Diffusion Samplingcited by 3
  18. 2024Gaussian Flow Bridges for Audio Domain Transfer with Unpaired Datacited by 3
  19. 2024HRTF Estimation using a Score-based Priorcited by 3
  20. 2025Token-Based Audio Inpainting via Discrete Diffusioncited by 3
  21. 2024ODE-DPS: ODE-based Diffusion Posterior Sampling for Inverse Problems in Partial Differential Equationcited by 2
  22. 2025Coupled Data and Measurement Space Dynamics for Enhanced Diffusion Posterior Samplingcited by 2
  23. 2025Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Modelcited by 2
  24. 2026Solving Room Impulse Response Inverse Problems Using Flow Matching with Analytic Wiener Denoisercited by 2
  25. 2024Audio Decoding by Inverse Problem Solvingcited by 1
  26. 2024Blind Spatial Impulse Response Generation from Separate Room- and Scene-Specific Informationcited by 1
  27. 2024Non-intrusive Speech Quality Assessment with Diffusion Models Trained on Clean Speechcited by 1
  28. 2025Towards HRTF Personalization using Denoising Diffusion Modelscited by 1
  29. 2025EquiReg: Equivariance Regularized Diffusion for Inverse Problemscited by 1
  30. 2025Diffusion-Based Unsupervised Audio-Visual Speech Separation in Noisy Environments with Noise Priorcited by 1
  31. 2025Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixturescited by 1
  32. 2026Outlier-Robust Diffusion Solvers for Inverse Problemscited by 1
  33. 2026Ambisonics Encoding of Room Impulse Responses using a Device-Agnostic Diffusion Modelcited by 1
  34. 2024Pictures Of MIDI: Controlled Music Generation via Graphical Prompts for Image-Based Diffusion Inpaintingcited by 0
  35. 2025SUBARU: A Practical Approach to Power Saving in Hearables Using SUB-Nyquist Audio Resolution Upsamplingcited by 0
  36. 2025Unsupervised Multi-channel Speech Dereverberation via Diffusioncited by 0
  37. 2025Similarity-Guided Diffusion for Long-Gap Music Inpaintingcited by 0
  38. 2025MACS: Measurement-Aware Consistency Sampling for Inverse Problemscited by 0

50 most cited of 62 All 62 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion-Based Audio Inpainting
  2. 2023EDMSound: Spectrogram Based Diffusion Models for Efficient and High-Quality Audio Synthesis
  3. 2025Post-Training Quantization for Audio Diffusion Transformers
  4. 2024DITTO: Diffusion Inference-Time T-Optimization for Music Generation
  5. 2026Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation
  6. 2024WaveTransfer: A Flexible End-to-end Multi-instrument Timbre Transfer with Diffusion
  7. 2023From Discrete Tokens to High-Fidelity Audio Using Multi-Band Diffusion
  8. 2025Audio Palette: A Diffusion Transformer with Multi-Signal Conditioning for Controllable Foley Synthesis