mlsanityData to 7 Oct 2026

Restoration based Generative Models

arXiv 2303.05456 ↗First posted 20 Feb 2023cs.LGcited by 6

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

Denoising diffusion models (DDMs) have recently attracted increasing attention by showing impressive synthesis quality. DDMs are built on a diffusion process that pushes data to the noise distribution and the models learn to denoise. In this paper, we establish the interpretation of DDMs in terms of image restoration (IR). Integrating IR literature allows us to use an alternative objective and diverse forward processes, not confining to the diffusion process. By imposing prior knowledge on the loss function grounded on MAP-based estimation, we eliminate the need for the expensive sampling of DDMs. Also, we propose a multi-scale training, which improves the performance compared to the diffusion process, by taking advantage of the flexibility of the forward process. Experimental results demonstrate that our model improves the quality and efficiency of both training and inference. Furthermore, we show the applicability of our model to inverse problems. We believe that our framework paves the way for designing a new type of flexible general generative model.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 66 of its 101 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

66 of 101 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Denoising Diffusion Restoration Models27 Jan

Other references · 65

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  6. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  7. 2022Generative Adversarial Networkscited by 5,603
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
38 more
  1. 2015Variational Inference with Normalizing Flowscited by 2,161
  2. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  3. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  4. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  5. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  6. 2021Variational Diffusion Modelscited by 1,145
  7. 2016Pixel Recurrent Neural Networkscited by 814
  8. 2022Pseudo Numerical Methods for Diffusion Models on Manifoldscited by 695
  9. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  10. 2021Score-based Generative Modeling in Latent Spacecited by 634
  11. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  12. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
  13. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
  14. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  15. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 515
  16. 2016The Little Engine that Could: Regularization by Denoising (RED)cited by 303
  17. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noisecited by 275
  18. 2015Generative Moment Matching Networkscited by 252
  19. 2017MMD GAN: Towards Deeper Understanding of Moment Matching Networkcited by 251
  20. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 230
  21. 2020Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulationcited by 225
  22. 2021Gotta Go Fast When Generating Data with Score-Based Modelscited by 218
  23. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  24. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  25. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  26. 2021On Fast Sampling of Diffusion Probabilistic Modelscited by 170
  27. 2022Generative Modelling With Inverse Heat Dissipationcited by 164
  28. 2020Adversarial Latent Autoencoderscited by 155
  29. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  30. 2015Training generative neural networks via Maximum Mean Discrepancy optimizationcited by 149
  31. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  32. 2020Learning Energy-Based Models by Diffusion Recovery Likelihoodcited by 122
  33. 2021Gradient Step Denoiser for convergent Plug-and-Playcited by 107
  34. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  35. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latentscited by 103
  36. 2018Adversarial Regularizers in Inverse Problemscited by 102
  37. 2018Regularization by Denoising: Clarifications and New Interpretationscited by 102
  38. 2022Soft Diffusion: Score Matching for General Corruptionscited by 101

50 most cited of 65 All 65 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 0; 2026: 1.

6 papers.

  1. 2023Generative Modeling through the Semi-dual Formulation of Unbalanced Optimal Transportcited by 39
  2. 2024Scalable Wasserstein Gradient Flow for Generative Modeling through Unbalanced Optimal Transportcited by 22
  3. 2024Residual-Conditioned Optimal Transport: Towards Structure-Preserving Unpaired and Paired Image Restorationcited by 13
  4. 2023Analyzing and Improving Optimal-Transport-based Adversarial Networkscited by 8
  5. 2024Scalable Simulation-free Entropic Unbalanced Optimal Transportcited by 5
  6. 2026One Pass Is Not Enough: Recursive Latent Refinement for Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancement
  2. 2023Residual Denoising Diffusion Models
  3. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
  4. 2024Taming Diffusion Models for Image Restoration: A Review
  5. 2023Restoration by Generation with Constrained Priors
  6. 2023Denoising Diffusion Bridge Models
  7. 2023Diffusion Model for Generative Image Denoising
  8. 2022Image Embedding for Denoising Generative Models