mlsanityData to 7 Oct 2026

Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariance

arXiv 2402.02149 ↗First posted 3 Feb 2024cs.CV cs.LGcited by 42

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Recent diffusion models provide a promising zero-shot solution to noisy linear inverse problems without retraining for specific inverse problems. In this paper, we reveal that recent methods can be uniformly interpreted as employing a Gaussian approximation with hand-crafted isotropic covariance for the intractable denoising posterior to approximate the conditional posterior mean. Inspired by this finding, we propose to improve recent methods by using more principled covariance determined by maximum likelihood estimation. To achieve posterior covariance optimization without retraining, we provide general plug-and-play solutions based on two approaches specifically designed for leveraging pre-trained models with and without reverse covariance. We further propose a scalable method for learning posterior covariance prediction based on representation with orthonormal basis. Experimental results demonstrate that the proposed methods significantly enhance reconstruction performance without requiring hyperparameter tuning.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 35 of its 44 references that are in the map. First posted on arXiv 3 Feb 2024.

Builds on

35 of 44 references.

Closest ancestors · 9

Same direction, newest first

  1. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusion1 Dec
  2. 2023Denoising Diffusion Models for Plug-and-Play Image Restoration15 May
  3. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Models7 May
  4. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imaging23 Apr
  5. 2023Refusion: Enabling Large-Size Realistic Image Restoration with Latent-Space Diffusion Models17 Apr
  6. 2023SUD2: Supervision by Denoising Diffusion Models for Image Reconstruction16 Mar
  7. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model1 Dec
  8. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep
  9. 2021Deblurring via Stochastic Refinement5 Dec

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  10. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  11. 2021Variational Diffusion Modelscited by 1,145
  12. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
14 more
  1. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  2. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  3. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  4. 2020Deep Unfolding Network for Image Super-Resolutioncited by 253
  5. 2020Nonlinear Transform Codingcited by 132
  6. 2018Taming VAEscited by 85
  7. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Modelscited by 83
  8. 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matchingcited by 79
  9. 2023Training-free Linear Image Inverses via Flowscited by 70
  10. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  11. 2023Frequency-Aware Transformer for Learned Image Compressioncited by 56
  12. 2023A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelscited by 37
  13. 2018Structured Uncertainty Prediction Networkscited by 34
  14. 2023Uncertainty Quantification via Neural Posterior Principal Componentscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 14; 2026: 15.

42 papers.

Sort:

By area:

  1. 2024Taming Rectified Flow for Inversion and Editingcited by 200
  2. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  3. 2024Steering Rectified Flow Models in the Vector Field for Controlled Image Generationcited by 38
  4. 2024Ensemble Kalman Diffusion Guidance: A Derivative-free Method for Inverse Problemscited by 18
  5. 2025Diffusion models for inverse problemscited by 15
  6. 2024Diffusion State-Guided Projected Gradient for Inverse Problemscited by 14
  7. 2024Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costscited by 14
  8. 2025IRBridge: Solving Image Restoration Bridge with Pre-trained Generative Diffusion Modelscited by 11
  9. 2024Inverse Problems with Diffusion Models: A MAP Estimation Perspectivecited by 9
  10. 2024Patch-Based Diffusion Models Beat Whole-Image Models for Mismatched Distribution Inverse Problemscited by 8
  11. 2024Equivariant Denoisers Cannot Copy Graphs: Align Your Graph Diffusion Modelscited by 7
  12. 2024Amortized Posterior Sampling with Diffusion Prior Distillationcited by 6
30 more
  1. 2024Gaussian is All You Need: A Unified Framework for Solving Inverse Problems via Diffusion Posterior Samplingcited by 5
  2. 2025Enhancing Diffusion Model Stability for Image Restoration via Gradient Managementcited by 5
  3. 2025PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inferencecited by 3
  4. 2026DM4CT: Benchmarking Diffusion Models for Computed Tomography Reconstructioncited by 3
  5. 2026Sample-efficient evidence estimation of score based priors for model selectioncited by 3
  6. 2024Bayesian Conditioned Diffusion Models for Inverse Problemscited by 2
  7. 2024Enhancing Diffusion Posterior Sampling for Inverse Problems by Integrating Crafted Measurementscited by 2
  8. 2025Conditional Mutual Information Based Diffusion Posterior Sampling for Solving Inverse Problemscited by 2
  9. 2025Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solvingcited by 2
  10. 2026Image Restoration via Diffusion Models with Dynamic Resolutioncited by 2
  11. 2025MAP-based Problem-Agnostic diffusion model for Inverse Problemscited by 1
  12. 2025EquiReg: Equivariance Regularized Diffusion for Inverse Problemscited by 1
  13. 2025Noise Conditional Variational Score Distillationcited by 1
  14. 2025Diffusion models for multivariate subsurface generation and efficient probabilistic inversioncited by 1
  15. 2025Inverse problems with diffusion models: MAP estimation via mode-seeking losscited by 1
  16. 2026Taming Score-Based Denoisers in ADMM: A Convergent Plug-and-Play Frameworkcited by 1
  17. 2026Covariance-aware sampling for Diffusion Modelscited by 1
  18. 2025Decomposing stimulus-specific sensory neural information via diffusion modelscited by 0
  19. 2025Jacobian-Aware Posterior Sampling for Inverse Problemscited by 0
  20. 2025Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equationscited by 0
  21. 2026Tracing the Oracle: Improving Diffusion Timestep Scheduling for 3D CT Reconstructioncited by 0
  22. 2026Stochastic Optimal Control Sampling for Diffusion Inverse Problemscited by 0
  23. 2026Concept Removal Guidance: Evidence-Calibrated Negative Guidance for Safe Diffusion Samplingcited by 0
  24. 2026Constrained Flow Matching via Lagrangian Dual Flowscited by 0
  25. 2026Posterior Information Dynamics of Diffusion Models for Linear Inverse Problemscited by 0
  26. 2026Direct Conditional Transition Sampling for Diffusion Inverse Problemscited by 0
  27. 2026Backward SDEs-based Diffusion for Physics-Constrained Generationcited by 0
  28. 2026ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problemscited by 0
  29. 2026AECSF: Adaptive Ensemble Conditional Score Filtering for High-Dimensional Nonlinear Data Assimilationcited by 0
  30. 2026Is your uncertainty map wrong, or is its target? Exact diagnostics for the Tweedie diagonal, and a gradient-free alternativecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Diffusion Models for Inverse Problems with Covariance-Aware Posterior Sampling
  2. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models
  3. 2026Noise-Adaptive Diffusion Sampling for Inverse Problems Without Task-Specific Tuning
  4. 2024A Survey on Diffusion Models for Inverse Problems
  5. 2026Learning Normalized Energy Models for Linear Inverse Problems
  6. 2024Unleashing the Denoising Capability of Diffusion Prior for Solving Inverse Problems
  7. 2026A Posterior-Dynamics Framework for Imaging Inverse Problems with Pretrained Diffusion Priors
  8. 2025A Recovery Theory for Diffusion Priors: Deterministic Analysis of the Implicit Prior Algorithm