mlsanityData to 7 Oct 2026

Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs

arXiv 2410.11149 ↗First posted 15 Oct 2024cs.LGcited by 14

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The covariance for clean data given a noisy observation is an important quantity in many training-free guided generation methods for diffusion models. Current methods require heavy test-time computation, altering the standard diffusion training process or denoiser architecture, or making heavy approximations. We propose a new framework that sidesteps these issues by using covariance information that is available for free from training data and the curvature of the generative trajectory, which is linked to the covariance through the second-order Tweedie's formula. We integrate these sources of information using (i) a novel method to transfer covariance estimates across noise levels and (ii) low-rank updates in a given noise level. We validate the method on linear inverse problems, where it outperforms recent baselines, especially with fewer diffusion steps.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 41 of its 58 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

41 of 58 references.

Closest ancestors · 13

Same direction, newest first

  1. 2023On the Posterior Distribution in Denoising: Application to Uncertainty Quantification24 Sep
  2. 2023User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systems13 Jun
  3. 2022Diffusion Models for Causal Discovery via Topological Ordering12 Oct
  4. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  5. 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching16 Jun
  6. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  8. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  9. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  10. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  11. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 28

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  5. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  6. 2022Denoising Diffusion Restoration Modelscited by 951
  7. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 574
  8. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  9. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  10. 2023Denoising Diffusion Models for Plug-and-Play Image Restorationcited by 335
  11. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 284
  12. 2021Deblurring via Stochastic Refinementcited by 248
16 more
  1. 2022Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problemcited by 229
  2. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 218
  3. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
  4. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  5. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  6. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 129
  7. 2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
  8. 2024D-Flow: Differentiating through Flows for Controlled Generationcited by 115
  9. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  10. 2021Estimating High Order Gradients of the Data Distribution by Denoisingcited by 63
  11. 2024Learning Diffusion Priors from Observations by Expectation Maximizationcited by 60
  12. 2024DMPlug: A Plug-in Method for Solving Inverse Problems with Diffusion Modelscited by 60
  13. 2024Zero-Shot Unsupervised and Text-Based Audio Editing Using DDPM Inversioncited by 58
  14. 2024Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariancecited by 42
  15. 2024Diffusion posterior sampling for simulation-based inference in tall data settingscited by 17
  16. 2024Taming Diffusion Models for Image Restoration: A Reviewcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 7.

14 papers.

Sort:

By area:

  1. 2025Progressive Tempering Sampler with Diffusioncited by 13
  2. 2025Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevincited by 5
  3. 2025PriorGuide: Test-Time Prior Adaptation for Simulation-Based Inferencecited by 3
  4. 2026Conditioning Gaussian Processes on Almost Anythingcited by 3
  5. 2025Guided Unconditional and Conditional Generative Models for Super-Resolution and Inference of Quasi-Geostrophic Turbulencecited by 2
  6. 2025Latent Refinement via Flow Matching for Training-free Linear Inverse Problem Solvingcited by 2
  7. 2026Covariance-aware sampling for Diffusion Modelscited by 1
  8. 2025Likelihood Matching for Diffusion Modelscited by 0
  9. 2025Softly Constrained Denoisers for Diffusion Models Applied to Partial Differential Equationscited by 0
  10. 2026Adaptive Moments are Surprisingly Effective for Plug-and-Play Diffusion Samplingcited by 0
  11. 2026Tempered Guided Diffusioncited by 0
  12. 2026Improved denoising diffusion probabilistic models with efficient non-diagonal covariance modelingcited by 0
2 more
  1. 2026ANaLOG: Anisotropic Native-Latent Operator Guidance for Solving Inverse Problemscited by 0
  2. 2026Is your uncertainty map wrong, or is its target? Exact diagnostics for the Tweedie diagonal, and a gradient-free alternativecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matching
  2. 2024On the Relation Between Linear Diffusion and Power Iteration
  3. 2025Learning What Matters: Steering Diffusion via Spectrally Anisotropic Forward Noise
  4. 2025Diffusion Models: A Mathematical Introduction
  5. 2025NoiseAR: AutoRegressing Initial Noise Prior for Diffusion Models
  6. 2025Kernel-Smoothed Scores for Denoising Diffusion: A Bias-Variance Study
  7. 2023Stable Target Field for Reduced Variance Score Estimation in Diffusion Models
  8. 2023Improving Denoising Diffusion Models via Simultaneous Estimation of Image and Noise