mlsanityData to 7 Oct 2026

Improving Diffusion Models for Inverse Problems using Manifold Constraints

arXiv 2206.00941 ↗First posted 2 Jun 2022cs.LG cs.AI cs.CVcited by 501

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Recently, diffusion models have been used to solve various inverse problems in an unsupervised manner with appropriate modifications to the sampling process. However, the current solvers, which recursively apply a reverse diffusion step followed by a projection-based measurement consistency step, often produce suboptimal results. By studying the generative sampling path, here we show that current solvers throw the sample path off the data manifold, and hence the error accumulates. To address this, we propose an additional correction term inspired by the manifold constraint, which can be used synergistically with the previous solvers to make the iterations close to the manifold. The proposed manifold constraint is straightforward to implement within a few lines of code, yet boosts the performance by a surprisingly large margin. With extensive experiments, we show that our method is superior to the previous methods both theoretically and empirically, producing promising results in many applications such as image inpainting, colorization, and sparse-view computed tomography. Code available https://github.com/HJ-harry/MCG_diffusion

Key facts. Cited by 501 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 33 of its 63 references that are in the map. First posted on arXiv 2 Jun 2022.

Builds on

33 of 63 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Denoising Diffusion Restoration Models27 Jan
  2. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contraction9 Dec

Other references · 31

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  10. 2022Video Diffusion Modelscited by 2,280
  11. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
19 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  2. 2016Least Squares Generative Adversarial Networkscited by 1,000
  3. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  4. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  5. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  6. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 407
  7. 2017Compressed Sensing using Generative Modelscited by 337
  8. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  9. 2019Guided Image Generation with Conditional Invertible Neural Networkscited by 147
  10. 2021High-Fidelity Pluralistic Image Completion with Transformerscited by 134
  11. 2021Noise2Score: Tweedie's Approach to Self-Supervised Image Denoising without Clean Imagescited by 110
  12. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  13. 2021Bayesian imaging using Plug & Play priors: when Langevin meets Tweediecited by 96
  14. 2021Aggregated Contextual Transformations for High-Resolution Image Inpaintingcited by 89
  15. 2016A deep convolutional neural network using directional wavelets for low-dose X-ray CT reconstructioncited by 87
  16. 2021Intermediate Layer Optimization for Inverse Problems using Deep Generative Modelscited by 70
  17. 2019Image-Adaptive GAN based Reconstructioncited by 36
  18. 20202-Step Sparse-View CT Reconstruction with a Domain-Specific Perceptual Networkcited by 9
  19. 2018Local Kernels that Approximate Bayesian Regularization and Proximal Operatorscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 24; 2023: 103; 2024: 167; 2025: 127; 2026: 80.

501 papers.

Sort:

By area:

  1. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  2. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  3. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  4. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 486
  5. 2023Universal Guidance for Diffusion Modelscited by 426
  6. 2023FreeDoM: Training-Free Energy-Guided Conditional Diffusion Modelcited by 290
  7. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  8. 2022A Survey on Generative Diffusion Modelcited by 238
  9. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  10. 2023Guided Motion Diffusion for Controllable Human Motion Synthesiscited by 221
  11. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  12. 2023I2SB: Image-to-Image Schr"odinger Bridgecited by 214
38 more
  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211
  3. 2023MotionDiffuser: Controllable Multi-Agent Motion Prediction using Diffusioncited by 210
  4. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistencycited by 180
  5. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  6. 2023Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restorationcited by 164
  7. 2024Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealingcited by 157
  8. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 156
  9. 2023Protein Design with Guided Discrete Diffusioncited by 154
  10. 2023Manifold Preserving Guided Diffusioncited by 153
  11. 2022Diffusion-based Image Translation using Disentangled Style and Content Representationcited by 148
  12. 2024CFG++: Manifold-constrained Classifier Free Guidance for Diffusion Modelscited by 138
  13. 2023Upscale-A-Video: Temporal-Consistent Diffusion Model for Real-World Video Super-Resolutioncited by 131
  14. 2022Solving 3D Inverse Problems using Pre-trained 2D Diffusion Modelscited by 130
  15. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 129
  16. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Modelscited by 128
  17. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  18. 2023Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometrycited by 121
  19. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 115
  20. 2022Parallel Diffusion Models of Operator and Image for Blind Inverse Problemscited by 113
  21. 2023SyncDiffusion: Coherent Montage via Synchronized Joint Diffusionscited by 108
  22. 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 108
  23. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101
  24. 2023DiffCollage: Parallel Generation of Large Content with Diffusion Modelscited by 100
  25. 2024DiffusionPDE: Generative PDE-Solving Under Partial Observationcited by 100
  26. 2023Optimizing Diffusion Noise Can Serve As Universal Motion Priorscited by 97
  27. 2022DifFace: Blind Face Restoration with Diffused Error Contractioncited by 96
  28. 2024Understanding Reinforcement Learning-Based Fine-Tuning of Diffusion Models: A Tutorial and Reviewcited by 92
  29. 2023Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Modelscited by 86
  30. 2024Arbitrary-steps Image Super-resolution via Diffusion Inversioncited by 85
  31. 2024NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizercited by 79
  32. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  33. 2022Multiscale Structure Guided Diffusion for Image Deblurringcited by 76
  34. 2022DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstructioncited by 73
  35. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  36. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  37. 2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
  38. 2023Image Denoising: The Deep Learning Revolution and Beyond -- A Survey Paper --cited by 71

50 most cited of 501 All 501 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems
  2. 2023Diffusion Models for Constrained Domains
  3. 2024Fast Samplers for Inverse Problems in Iterative Refinement Models
  4. 2026The Right Measure for Physics-Constrained Generation: A Co-Area Correction for Posterior-Consistent PDE Inverse Problems
  5. 2025Diffusion Models for Solving Inverse Problems via Posterior Sampling with Piecewise Guidance
  6. 2023Metropolis Sampling for Constrained Diffusion Models
  7. 2026Manifold-Constrained Noise Optimization for Diverse Diffusion Sampling
  8. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions