mlsanityData to 7 Oct 2026

A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Models

arXiv 2302.01217 ↗First posted 2 Feb 2023stat.ML cs.AI cs.LGcited by 37

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We provide a theoretical justification for sample recovery using diffusion based image inpainting in a linear model setting. While most inpainting algorithms require retraining with each new mask, we prove that diffusion based inpainting generalizes well to unseen masks without retraining. We analyze a recently proposed popular diffusion based inpainting algorithm called RePaint (Lugmayr et al., 2022), and show that it has a bias due to misalignment that hampers sample recovery even in a two-state diffusion process. Motivated by our analysis, we propose a modified RePaint algorithm we call RePaint^+ that provably recovers the underlying true sample and enjoys a linear rate of convergence. It achieves this by rectifying the misalignment error present in drift and dispersion of the reverse process. To the best of our knowledge, this is the first linear convergence result for a diffusion based image inpainting algorithm.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 36 of its 50 references that are in the map. First posted on arXiv 2 Feb 2023.

Builds on

36 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Models24 Jan

Other references · 35

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  11. 2016Density estimation using Real NVPcited by 2,029
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
23 more
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  2. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  3. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
  4. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  5. 2022Denoising Diffusion Restoration Modelscited by 951
  6. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 634
  7. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  8. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
  9. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  10. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  11. 2017Compressed Sensing using Generative Modelscited by 337
  12. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 284
  13. 2021Deblurring via Stochastic Refinementcited by 248
  14. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 237
  15. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptionscited by 215
  16. 2022Convergence of score-based generative modeling for general data distributionscited by 179
  17. 2022Convergence for score-based generative modeling with polynomial complexitycited by 176
  18. 2022Score-Based Generative Models Detect Manifoldscited by 121
  19. 2021Generating Diverse Structure for Image Inpainting With Hierarchical VQ-VAEcited by 106
  20. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 95
  21. 2019Deep Compressed Sensingcited by 46
  22. 2018Modeling Sparse Deviations for Compressed Sensing using Generative Modelscited by 33
  23. 2020Robust Compressed Sensing using Generative Modelscited by 33

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 18; 2025: 2; 2026: 2.

37 papers.

Sort:

By area:

  1. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  2. 2024Semantic Image Inversion and Editing using Rectified Stochastic Differential Equationscited by 162
  3. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 156
  4. 2023Decomposed Diffusion Sampler for Accelerating Large-Scale Inverse Problemscited by 129
  5. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
  6. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Modelscited by 84
  7. 2023Beyond First-Order Tweedie: Solving Inverse Problems using Latent Diffusioncited by 78
  8. 2023Non-autoregressive Conditional Diffusion Models for Time Series Predictioncited by 77
  9. 2024All-in-one simulation-based inferencecited by 71
  10. 2024Guidance with Spherical Gaussian Constraint for Conditional Diffusioncited by 69
  11. 2023Residual Denoising Diffusion Modelscited by 68
  12. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Modelscited by 64
25 more
  1. 2023Towards Coherent Image Inpainting Using Denoising Diffusion Implicit Modelscited by 61
  2. 2023Unsupervised Out-of-Distribution Detection with Diffusion Inpaintingcited by 57
  3. 2024RB-Modulation: Training-Free Personalization of Diffusion Models using Stochastic Optimal Controlcited by 46
  4. 2024Improving Diffusion Models for Inverse Problems Using Optimal Posterior Covariancecited by 42
  5. 2024SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesiscited by 34
  6. 2023Accurate transition state generation with an object-aware equivariant elementary reaction diffusion modelcited by 30
  7. 2023User-defined Event Sampling and Uncertainty Quantification in Diffusion Models for Physical Dynamical Systemscited by 30
  8. 2024Be-Your-Outpainter: Mastering Video Outpainting through Input-Specific Adaptationcited by 29
  9. 2024Distribution-Aware Data Expansion with Diffusion Modelscited by 20
  10. 2023PMAA: A Progressive Multi-scale Attention Autoencoder Model for High-performance Cloud Removal from Multi-temporal Satellite Imagerycited by 16
  11. 2023Masked Modeling for Self-supervised Representation Learning on Vision and Beyondcited by 13
  12. 2024Resisting Stochastic Risks in Diffusion Planners with the Trajectory Aggregation Treecited by 13
  13. 2023Squeezing Large-Scale Diffusion Models for Mobilecited by 11
  14. 2024Expensive Multi-Objective Bayesian Optimization Based on Diffusion Modelscited by 10
  15. 2024TMPQ-DM: Joint Timestep Reduction and Quantization Precision Selection for Efficient Diffusion Modelscited by 9
  16. 2024Frequency-Time Diffusion with Neural Cellular Automatacited by 8
  17. 2024DISCO: Language-Guided Manipulation with Diffusion Policies and Constrained Inpaintingcited by 8
  18. 2025Diffusion LLMs are Natural Adversaries for any LLMcited by 7
  19. 2024Constrained Posterior Sampling: Time Series Generation with Hard Constraintscited by 6
  20. 2023Local-to-Global Panorama Inpainting for Locale-Aware Indoor Lighting Predictioncited by 3
  21. 2025A Recovery Theory for Diffusion Priors: Deterministic Analysis of the Implicit Prior Algorithmcited by 3
  22. 2024Varying Manifolds in Diffusion: From Time-varying Geometries to Visual Saliencycited by 2
  23. 2024Neuroexplicit Diffusion Models for Inpainting of Optical Flow Fieldscited by 0
  24. 2026Generative diffusion models for spatiotemporal influenza forecastingcited by 0
  25. 2026Flow-Based Generative Modeling for Optimizing Sampling Policies in Compressed Sensing Applicationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Gradpaint: Gradient-Guided Inpainting with Diffusion Models
  2. 20243D-Consistent Image Inpainting with Diffusion Models
  3. 2024Minutes to Seconds: Speeded-up DDPM-based Image Inpainting with Coarse-to-Fine Sampling
  4. 2024Diffusion-based image inpainting with internal learning
  5. 2021Learning Sparse Masks for Diffusion-based Image Inpainting
  6. 2024VIPaint: Image Inpainting with Pre-Trained Diffusion Models via Variational Inference
  7. 2025LanPaint: Training-Free Diffusion Inpainting with Asymptotically Exact and Fast Conditional Sampling
  8. 2024RAD: Region-Aware Diffusion Models for Image Inpainting