# Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise

arXiv: https://arxiv.org/abs/2311.14900 · HTML: https://mlsanity.com/paper/2311.14900/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models.
- Builds on 48 of its 65 references that are in the map.
- First posted on arXiv 25 Nov 2023.

## Placed in

- Diffusion models / [Inverse problems and image restoration](https://mlsanity.com/diffusion/inverse-problems-and-image-restoration/)

## Builds on (most cited first, 30 of 48)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023)
- [Kindling the Darkness: A Practical Low-light Image Enhancer](https://mlsanity.com/paper/1905.04161/) (2019)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022)
- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [I2SB: Image-to-Image Schr"odinger Bridge](https://mlsanity.com/paper/2302.05872/) (2023)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [SNIPS: Solving Noisy Inverse Problems Stochastically](https://mlsanity.com/paper/2105.14951/) (2021)
- [Low-Light Image Enhancement with Normalizing Flow](https://mlsanity.com/paper/2109.05923/) (2021)

## Built on by (most cited first, 19 of 19)

- [Diffusion Models in Low-Level Vision: A Survey](https://mlsanity.com/paper/2406.11138/) (2024)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [Residual Diffusion Bridge Model for Image Restoration](https://mlsanity.com/paper/2510.23116/) (2025)
- [Reversing Flow for Image Restoration](https://mlsanity.com/paper/2506.16961/) (2025)
- [Optimizing for the Shortest Path in Denoising Diffusion Model](https://mlsanity.com/paper/2503.03265/) (2025)
- [Kernel Density Steering: Inference-Time Scaling via Mode Seeking for Image Restoration](https://mlsanity.com/paper/2507.05604/) (2025)
- [FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restoration](https://mlsanity.com/paper/2511.14099/) (2025)
- [Efficient Image Restoration with State-Dependent Forward Diffusion](https://mlsanity.com/paper/2505.16733/) (2025)
- [R3D: Regional-guided Residual Radar Diffusion](https://mlsanity.com/paper/2601.06465/) (2026)
- [A Simple Combination of Diffusion Models for Better Quality Trade-Offs in Image Denoising](https://mlsanity.com/paper/2503.14654/) (2025)
- [Residual Prior Diffusion: A Probabilistic Framework Integrating Coarse Latent Priors with Diffusion Models](https://mlsanity.com/paper/2512.21593/) (2025)
- [Breaking Watermarks in the Frequency Domain: A Modulated Diffusion Attack Framework](https://mlsanity.com/paper/2604.22220/) (2026)
- [Fake3DGS: A Benchmark for 3D Manipulation Detection in Neural Rendering](https://mlsanity.com/paper/2604.27590/) (2026)
- [Two-stage deep learning framework for the restoration of incomplete-ring PET images](https://mlsanity.com/paper/2504.00816/) (2025)
- [DHO2: Accelerating Distributed Hybrid Order Optimization via Model Parallelism and ADMM](https://mlsanity.com/paper/2505.00982/) (2025)
- [Double-Diffusion: Balancing Speed, Accuracy, and Uncertainty in Probabilistic Forecasting for Urban Sensor Networks](https://mlsanity.com/paper/2506.23053/) (2025)
- [Structure-constrained Language-informed Diffusion Model for Unpaired Low-dose Computed Tomography Angiography Reconstruction](https://mlsanity.com/paper/2601.20304/) (2026)
- [IR-Flow: Bridging Discriminative and Generative Image Restoration via Rectified Flow](https://mlsanity.com/paper/2604.19680/) (2026)
- [ScaleResfusion: Residual Rectified Flow based on Residual Vector Field](https://mlsanity.com/paper/2607.25275/) (2026)

## Abstract (arXiv)

Recently, research on denoising diffusion models has expanded its application to the field of image restoration. Traditional diffusion-based image restoration methods utilize degraded images as conditional input to effectively guide the reverse generation process, without modifying the original denoising diffusion process. However, since the degraded images already include low-frequency information, starting from Gaussian white noise will result in increased sampling steps. We propose Resfusion, a general framework that incorporates the residual term into the diffusion forward process, starting the reverse process directly from the noisy degraded images. The form of our inference process is consistent with the DDPM. We introduced a weighted residual noise, named resnoise, as the prediction target and explicitly provide the quantitative relationship between the residual term and the noise term in resnoise. By leveraging a smooth equivalence transformation, Resfusion determine the optimal acceleration step and maintains the integrity of existing noise schedules, unifying the training and inference processes. The experimental results demonstrate that Resfusion exhibits competitive performance on ISTD dataset, LOL dataset and Raindrop dataset with only five sampling steps. Furthermore, Resfusion can be easily applied to image generation and emerges with strong versatility. Our code and model are available at https://github.com/nkicsl/Resfusion.