mlsanityData to 7 Oct 2026

Input Perturbation Reduces Exposure Bias in Diffusion Models

arXiv 2301.11706 ↗First posted 27 Jan 2023cs.LG cs.AI cs.CVcited by 88

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Denoising Diffusion Probabilistic Models have shown an impressive generation quality, although their long sampling chain leads to high computational costs. In this paper, we observe that a long sampling chain also leads to an error accumulation phenomenon, which is similar to the exposure bias problem in autoregressive text generation. Specifically, we note that there is a discrepancy between training and testing, since the former is conditioned on the ground truth samples, while the latter is conditioned on the previously generated results. To alleviate this problem, we propose a very simple but effective training regularization, consisting in perturbing the ground truth samples to simulate the inference time prediction errors. We empirically show that, without affecting the recall and precision, the proposed input perturbation leads to a significant improvement in the sample quality while reducing both the training and the inference times. For instance, on CelebA 64×64, we achieve a new state-of-the-art FID score of 1.27, while saving 37.5% of the training time. The code is publicly available at https://github.com/forever208/DDPM-IP

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 46 of its 58 references that are in the map. First posted on arXiv 27 Jan 2023.

Builds on

46 of 58 references.

Closest ancestors · 13

Same direction, newest first

  1. 2022Diffusion Models in Vision: A Survey10 Sep
  2. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  3. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  7. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  8. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  12. 2020Denoising Diffusion Implicit Models6 Oct
1 more
  1. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 33

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  11. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
21 more
  1. 2017Improved Training of Wasserstein GANscited by 2,223
  2. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  3. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  4. 2017Mixed Precision Trainingcited by 1,135
  5. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  6. 2015Scheduled Sampling for Sequence Prediction with Recurrent Neural Networkscited by 779
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  8. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
  9. 2021Argmax Flows and Multinomial Diffusion: Learning Categorical Distributionscited by 566
  10. 2016Self-critical Sequence Training for Image Captioningcited by 544
  11. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  12. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  13. 2021Generating Images with Sparse Representationscited by 262
  14. 2021Autoregressive Diffusion Modelscited by 208
  15. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  16. 2021Symbolic Music Generation with Diffusion Modelscited by 152
  17. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  18. 2021Efficient Training of Visual Transformers with Small Datasetscited by 107
  19. 2022Poisson Flow Generative Modelscited by 95
  20. 2022Learning Smooth Neural Functions via Lipschitz Regularizationcited by 86
  21. 2022The Effects of Regularization and Data Augmentation are Class Dependentcited by 67

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 16; 2024: 26; 2025: 21; 2026: 25.

88 papers.

Sort:

By area:

  1. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 98
  2. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 87
  3. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Stepscited by 71
  4. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 59
  5. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
  6. 2024Learning to Discretize Denoising Diffusion ODEscited by 39
  7. 2023Exploiting the Signal-Leak Bias in Diffusion Modelscited by 36
  8. 2023Quantum circuit synthesis with diffusion modelscited by 33
  9. 2024DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Spacecited by 32
  10. 2023On Error Propagation of Diffusion Modelscited by 30
  11. 2024DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecturecited by 21
  12. 2025Noise-Consistent Siamese-Diffusion for Medical Image Synthesis and Segmentationcited by 21
38 more
  1. 2023The Devil's Advocate: Shattering the Illusion of Unexploitable Data using Diffusion Modelscited by 19
  2. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 16
  3. 2024Diffusion Posterior Proximal Sampling for Image Restorationcited by 14
  4. 2024Cinemo: Consistent and Controllable Image Animation with Motion Diffusion Modelscited by 14
  5. 2023Unmasking Bias in Diffusion Model Trainingcited by 13
  6. 2024Timestep-Aware Correction for Quantized Diffusion Modelscited by 13
  7. 2023DREAM: Diffusion Rectification and Estimation-Adaptive Modelscited by 12
  8. 2024Representation Learning and Identity Adversarial Training for Facial Behavior Understandingcited by 12
  9. 2023Distilling ODE Solvers of Diffusion Models into Smaller Stepscited by 11
  10. 2024Structure-Guided Adversarial Training of Diffusion Modelscited by 11
  11. 2024NeuroCine: Decoding Vivid Video Sequences from Human Brain Activtiescited by 10
  12. 2024Animate Your Motion: Turning Still Images into Dynamic Videoscited by 10
  13. 2023Precision-Recall Divergence Optimization for Generative Modeling with GANs and Normalizing Flowscited by 9
  14. 2023Synthetic Shifts to Initial Seed Vector Exposes the Brittle Nature of Latent-Based Diffusion Modelscited by 9
  15. 2024Slight Corruption in Pre-training Data Makes Better Diffusion Modelscited by 9
  16. 2025Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Modelscited by 9
  17. 2026Persistent Robot World Models: Stabilizing Multi-Step Rollouts via Reinforcement Learningcited by 9
  18. 2023Importance of Smoothness Induced by Optimizers in FL4ASR: Towards Understanding Federated Learning for End-to-End ASRcited by 8
  19. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  20. 2025Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compressioncited by 7
  21. 2023Training Energy-Based Normalizing Flow with Score-Matching Objectivescited by 6
  22. 2026Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusioncited by 6
  23. 2026SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Modelscited by 6
  24. 2024Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Modelscited by 5
  25. 2024Illustrious: an Open Advanced Illustration Modelcited by 5
  26. 2025Frequency Regulation for Exposure Bias Mitigation in Diffusion Modelscited by 5
  27. 2024Enhancing Sample Generation of Diffusion Models using Noise Level Correctioncited by 4
  28. 2025ADT: Tuning Diffusion Models with Adversarial Supervisioncited by 4
  29. 2025Aligning Generative Denoising with Discriminative Objectives Unleashes Diffusion for Visual Perceptioncited by 4
  30. 2025How I Met Your Bias: Investigating Bias Amplification in Diffusion Modelscited by 4
  31. 2026Continuous-Time Distribution Matching for Few-Step Diffusion Distillationcited by 4
  32. 2026Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matchingcited by 4
  33. 2025Improved Diffusion-based Generative Model with Better Adversarial Robustnesscited by 3
  34. 2025Understanding Flatness in Generative Models: Its Role and Benefitscited by 3
  35. 2025Consistent and Controllable Image Animation with Motion Linear Diffusion Transformerscited by 3
  36. 2026Elucidating the SNR-t Bias of Diffusion Probabilistic Modelscited by 3
  37. 2023Deshadow-Anything: When Segment Anything Model Meets Zero-shot shadow removalcited by 2
  38. 2023Mitigating Exposure Bias in Discriminator Guided Diffusion Modelscited by 2

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Can Diffusion Model Achieve Better Performance in Text Generation? Bridging the Gap between Training and Inference!
  2. 2026Is Your Diffusion Sampler Actually Correct? A Sampler-Centric Evaluation of Discrete Diffusion Language Models
  3. 2025Improving Diffusion Generative Models via Truncated Karhunen--Lo`eve Expansion
  4. 2024Evaluating the design space of diffusion-based generative models
  5. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  6. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps
  7. 2025Is Your Conditional Diffusion Model Actually Denoising?
  8. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models