# Improving Diffusion Inverse Problem Solving with Decoupled Noise Annealing

arXiv: https://arxiv.org/abs/2407.01521 · HTML: https://mlsanity.com/paper/2407.01521/

## Key facts

- Cited by 157 papers in the map (data through 7 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 45 of its 60 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 45)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Diffusion Posterior Sampling for General Noisy Inverse Problems](https://mlsanity.com/paper/2209.14687/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [Solving Inverse Problems in Medical Imaging with Score-Based Generative Models](https://mlsanity.com/paper/2111.08005/) (2021)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [fastMRI: An Open Dataset and Benchmarks for Accelerated MRI](https://mlsanity.com/paper/1811.08839/) (2018)
- [A Continuous Time Framework for Discrete Denoising Models](https://mlsanity.com/paper/2205.14987/) (2022)
- [Analyzing and Improving the Training Dynamics of Diffusion Models](https://mlsanity.com/paper/2312.02696/) (2023)
- [Denoising Diffusion Models for Plug-and-Play Image Restoration](https://mlsanity.com/paper/2305.08995/) (2023)
- [Score-based diffusion models for accelerated MRI](https://mlsanity.com/paper/2110.05243/) (2021)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Plug-and-Play ADMM for Image Restoration: Fixed Point Convergence and Applications](https://mlsanity.com/paper/1605.01710/) (2016)
- [A Variational Perspective on Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2305.04391/) (2023)
- [SNIPS: Solving Noisy Inverse Problems Stochastically](https://mlsanity.com/paper/2105.14951/) (2021)
- [Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistency](https://mlsanity.com/paper/2307.08123/) (2023)
- [Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models](https://mlsanity.com/paper/2307.00619/) (2023)
- [Score-Based Diffusion Models as Principled Priors for Inverse Imaging](https://mlsanity.com/paper/2304.11751/) (2023)

## Built on by (most cited first, 30 of 157)

- [PnP-Flow: Plug-and-Play Image Restoration with Flow Matching](https://mlsanity.com/paper/2410.02423/) (2024)
- [InverseBench: Benchmarking Plug-and-Play Diffusion Priors for Inverse Problems in Physical Sciences](https://mlsanity.com/paper/2503.11043/) (2025)
- [FlowDPS: Flow-Driven Posterior Sampling for Inverse Problems](https://mlsanity.com/paper/2503.08136/) (2025)
- [Sequential Controlled Langevin Diffusions](https://mlsanity.com/paper/2412.07081/) (2024)
- [Guided Diffusion Sampling on Function Spaces with Applications to PDEs](https://mlsanity.com/paper/2505.17004/) (2025)
- [Energy Matching: Unifying Flow Matching and Energy-Based Models for Generative Modeling](https://mlsanity.com/paper/2504.10612/) (2025)
- [SITCOM: Step-wise Triple-Consistent Diffusion Sampling for Inverse Problems](https://mlsanity.com/paper/2410.04479/) (2024)
- [Flower: A Flow-Matching Solver for Inverse Problems](https://mlsanity.com/paper/2509.26287/) (2025)
- [Rethinking Diffusion Posterior Sampling: From Conditional Score Estimator to Maximizing a Posterior](https://mlsanity.com/paper/2501.18913/) (2025)
- [Split Gibbs Discrete Diffusion Posterior Sampling](https://mlsanity.com/paper/2503.01161/) (2025)
- [Solving Inverse Problems with FLAIR](https://mlsanity.com/paper/2506.02680/) (2025)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [Diffusion State-Guided Projected Gradient for Inverse Problems](https://mlsanity.com/paper/2410.03463/) (2024)
- [Residual Diffusion Bridge Model for Image Restoration](https://mlsanity.com/paper/2510.23116/) (2025)
- [Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation](https://mlsanity.com/paper/2505.22391/) (2025)
- [PostEdit: Posterior Sampling for Efficient Zero-Shot Image Editing](https://mlsanity.com/paper/2410.04844/) (2024)
- [Solving Linear-Gaussian Bayesian Inverse Problems with Decoupled Diffusion Sequential Monte Carlo](https://mlsanity.com/paper/2502.06379/) (2025)
- [InvFusion: Bridging Supervised and Zero-shot Diffusion for Inverse Problems](https://mlsanity.com/paper/2504.01689/) (2025)
- [Decoupled Diffusion Sampling for Inverse Problems on Function Spaces](https://mlsanity.com/paper/2601.23280/) (2026)
- [Multiscale guidance of protein structure prediction with heterogeneous cryo-EM data](https://mlsanity.com/paper/2506.04490/) (2025)
- [Inverse Problem Sampling in Latent Space Using Sequential Monte Carlo](https://mlsanity.com/paper/2502.05908/) (2025)
- [Steering Generative Models with Experimental Data for Protein Fitness Optimization](https://mlsanity.com/paper/2505.15093/) (2025)
- [Integrating Intermediate Layer Optimization and Projected Gradient Descent for Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2505.20789/) (2025)
- [Whitened Score Diffusion: A Structured Prior for Imaging Inverse Problems](https://mlsanity.com/paper/2505.10311/) (2025)
- [ESS-Flow: Training-free guidance of flow-based models as inference in source space](https://mlsanity.com/paper/2510.05849/) (2025)
- [ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problems](https://mlsanity.com/paper/2506.10955/) (2025)
- [Test-Time Anchoring for Discrete Diffusion Posterior Sampling](https://mlsanity.com/paper/2510.02291/) (2025)
- [FlowLPS: Langevin-Proximal Sampling for Flow-based Inverse Problem Solvers](https://mlsanity.com/paper/2512.07150/) (2025)
- [Efficient Zero-Shot Inpainting with Decoupled Diffusion Guidance](https://mlsanity.com/paper/2512.18365/) (2025)
- [D-Flow SGLD: Source-Space Posterior Sampling for Scientific Inverse Problems with Flow Matching](https://mlsanity.com/paper/2602.21469/) (2026)

## Abstract (arXiv)

Diffusion models have recently achieved success in solving Bayesian inverse problems with learned data priors. Current methods build on top of the diffusion sampling process, where each denoising step makes small modifications to samples from the previous step. However, this process struggles to correct errors from earlier sampling steps, leading to worse performance in complicated nonlinear inverse problems, such as phase retrieval. To address this challenge, we propose a new method called Decoupled Annealing Posterior Sampling (DAPS) that relies on a novel noise annealing process. Specifically, we decouple consecutive steps in a diffusion sampling trajectory, allowing them to vary considerably from one another while ensuring their time-marginals anneal to the true posterior as we reduce noise levels. This approach enables the exploration of a larger solution space, improving the success rate for accurate reconstructions. We demonstrate that DAPS significantly improves sample quality and stability across multiple image restoration tasks, particularly in complicated nonlinear inverse problems.