mlsanityData to 7 Oct 2026

Compensation Sampling for Improved Convergence in Diffusion Models

arXiv 2312.06285 ↗First posted 11 Dec 2023cs.CV cs.AIcited by 1

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models achieve remarkable quality in image generation, but at a cost. Iterative denoising requires many time steps to produce high fidelity images. We argue that the denoising process is crucially limited by an accumulation of the reconstruction error due to an initial inaccurate reconstruction of the target data. This leads to lower quality outputs, and slower convergence. To address this issue, we propose compensation sampling to guide the generation towards the target domain. We introduce a compensation term, implemented as a U-Net, which adds negligible computation overhead during training and, optionally, inference. Our approach is flexible and we demonstrate its application in unconditional generation, face inpainting, and face de-occlusion using benchmark datasets CIFAR-10, CelebA, CelebA-HQ, FFHQ-256, and FSG. Our approach consistently yields state-of-the-art results in terms of image quality, while accelerating the denoising process to converge during training by up to an order of magnitude.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 53 of its 63 references that are in the map. First posted on arXiv 11 Dec 2023.

Builds on

53 of 63 references.

Closest ancestors · 23

Same direction, newest first

  1. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Models25 Apr
  2. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  3. 2023Stable Target Field for Reduced Variance Score Estimation in Diffusion Models1 Feb
  4. 2022On Distillation of Guided Diffusion Models6 Oct
  5. 2022Soft Diffusion: Score Matching for General Corruptions12 Sep
  6. 2022Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise19 Aug
  7. 2022Entropy-driven Sampling and Training Scheme for Conditional Diffusion Generation23 Jun
  8. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  9. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2022Perception Prioritized Training of Diffusion Models1 Apr
  12. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
11 more
  1. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  2. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latents2 Jan
  3. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  4. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representation30 Nov
  5. 2021Conditional Image Generation with Score-Based Diffusion Models26 Nov
  6. 2021Score-based Generative Modeling in Latent Space10 Jun
  7. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation10 Jun
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  10. 2020Denoising Diffusion Implicit Models6 Oct
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 30

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  12. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
18 more
  1. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  2. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  3. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  4. 2018Generative Image Inpainting with Contextual Attentioncited by 578
  5. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  6. 2021Generating Images with Sparse Representationscited by 262
  7. 2021Large Scale Image Completion via Co-Modulated Generative Adversarial Networkscited by 219
  8. 2022MAT: Mask-Aware Transformer for Large Hole Image Inpaintingcited by 216
  9. 2022Diffusion-GAN: Training GANs with Diffusioncited by 203
  10. 2023Generative Diffusion Prior for Unified Image Restoration and Enhancementcited by 198
  11. 2020A U-Net Based Discriminator for Generative Adversarial Networkscited by 128
  12. 2019Generative Teaching Networks: Accelerating Neural Architecture Search by Learning to Generate Synthetic Training Datacited by 103
  13. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  14. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Imagescited by 74
  15. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  16. 2022Keys to Better Image Inpainting: Structure and Texture Go Hand in Handcited by 25
  17. 2022GLaMa: Joint Spatial and Frequency Loss for General Image Inpaintingcited by 15
  18. 2021Segmentation-Reconstruction-Guided Facial Image De-occlusioncited by 6

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024RAD: Region-Aware Diffusion Models for Image Inpaintingcited by 19

Similar, not cited

Close in wording, no citation either way

  1. 2024Diffusion Posterior Proximal Sampling for Image Restoration
  2. 2025Error Propagation Mechanisms and Compensation Strategies for Quantized Diffusion
  3. 2025LowDiff: Efficient Diffusion Sampling with Low-Resolution Condition
  4. 2024Enhancing Sample Generation of Diffusion Models using Noise Level Correction
  5. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision
  6. 2025It Takes Two to Tango: Two Parallel Samplers Improve Quality in Diffusion Models for Limited Steps
  7. 2025Training-free Diffusion Acceleration with Bottleneck Sampling
  8. 2023The Missing U for Efficient Diffusion Models