mlsanityData to 14 Oct 2026

Elucidating the Exposure Bias in Diffusion Models

arXiv 2308.15321 ↗First posted 29 Aug 2023cs.LG cs.AI cs.CVcited by 101

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have demonstrated impressive generative capabilities, but their textitexposure bias problem, described as the input mismatch between training and sampling, lacks in-depth exploration. In this paper, we systematically investigate the exposure bias problem in diffusion models by first analytically modelling the sampling distribution, based on which we then attribute the prediction error at each sampling step as the root cause of the exposure bias issue. Furthermore, we discuss potential solutions to this issue and propose an intuitive metric for it. Along with the elucidation of exposure bias, we propose a simple, yet effective, training-free method called Epsilon Scaling to alleviate the exposure bias. We show that Epsilon Scaling explicitly moves the sampling trajectory closer to the vector field learned in the training phase by scaling down the network output, mitigating the input mismatch between training and sampling. Experiments on various diffusion frameworks (ADM, DDIM, EDM, LDM, DiT, PFGM++) verify the effectiveness of our method. Remarkably, our ADM-ES, as a state-of-the-art stochastic sampler, obtains 2.17 FID on CIFAR-10 under 100-step unconditional generation. The code is available at urlhttps://github.com/forever208/ADM-ES and urlhttps://github.com/forever208/EDM-ES.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 48 of its 59 references that are in the map. First posted on arXiv 29 Aug 2023.

Builds on

48 of 59 references.

Closest ancestors · 21

Same direction, newest first

  1. 2023On Error Propagation of Diffusion Models9 Aug
  2. 2023Restart Sampling for Improving Generative Processes26 Jun
  3. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps24 May
  4. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  5. 2023Input Perturbation Reduces Exposure Bias in Diffusion Models27 Jan
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022Classifier-Free Diffusion Guidance26 Jul
  8. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Models15 Jun
  9. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  11. 2022Dynamic Dual-Output Diffusion Models8 Mar
  12. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
9 more
  1. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANs15 Dec
  4. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  5. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  6. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Implicit Models6 Oct
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 27

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  7. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  9. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  10. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  11. 2016Density estimation using Real NVPcited by 2,042
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
15 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  3. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  4. 2014NICE: Non-linear Independent Components Estimationcited by 1,058
  5. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  6. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  8. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  9. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  10. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  11. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 337
  12. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 259
  13. 2019Generalization in Generation: A closer look at Exposure Biascited by 123
  14. 2022Poisson Flow Generative Modelscited by 95
  15. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 16; 2025: 43; 2026: 39.

101 papers.

Sort:

By area:

  1. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
  2. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 89
  3. 2024DPLM-2: A Multimodal Diffusion Protein Language Modelcited by 64
  4. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 52
  5. 2023Fair Text-to-Image Diffusion via Fair Mappingcited by 31
  6. 2026ShotStream: Streaming Multi-Shot Video Generation for Interactive Storytellingcited by 28
  7. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 25
  8. 2024Score-based Generative Diffusion Models for Social Recommendationscited by 23
  9. 2025Macro-from-Micro Planning for High-Quality and Parallelized Autoregressive Long Video Generationcited by 21
  10. 2024Boosting Latent Diffusion with Perceptual Objectivescited by 20
  11. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 17
  12. 2025BAgger: Backwards Aggregation for Mitigating Drift in Autoregressive Video Diffusion Modelscited by 16
38 more
  1. 2024Timestep-Aware Correction for Quantized Diffusion Modelscited by 13
  2. 2025Distinguished Quantized Guidance for Diffusion-based Sequence Recommendationcited by 13
  3. 2025Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Modelscited by 13
  4. 2026Beyond Pixel Histories: World Models with Persistent 3D Statecited by 13
  5. 2023DREAM: Diffusion Rectification and Estimation-Adaptive Modelscited by 12
  6. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  7. 2024Diffusion Rejection Samplingcited by 11
  8. 2025Learning to See in the Extremely Darkcited by 11
  9. 2024NeuroCine: Decoding Vivid Video Sequences from Human Brain Activtiescited by 10
  10. 2024Animate Your Motion: Turning Still Images into Dynamic Videoscited by 10
  11. 2025EdiVal-Agent: An Object-Centric Framework for Automated, Fine-Grained Evaluation of Multi-Turn Editingcited by 10
  12. 2026Causal Motion Diffusion Models for Autoregressive Motion Generationcited by 10
  13. 2024Slight Corruption in Pre-training Data Makes Better Diffusion Modelscited by 9
  14. 2025Autoregressive Distillation of Diffusion Transformerscited by 9
  15. 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 9
  16. 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 9
  17. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  18. 2025ISPDiffuser: Learning RAW-to-sRGB Mappings with Texture-Aware Diffusion Models and Histogram-Guided Color Consistencycited by 7
  19. 2026Spectrum Matching: a Unified Perspective for Superior Diffusability in Latent Diffusioncited by 7
  20. 2025Proxy-Tuning: Tailoring Multimodal Autoregressive Models for Subject-Driven Image Generationcited by 6
  21. 2025Frequency Regulation for Exposure Bias Mitigation in Diffusion Modelscited by 6
  22. 2025DMQ: Dissecting Outliers of Diffusion Models for Post-Training Quantizationcited by 6
  23. 2025Adaptive Stochastic Coefficients for Accelerating Diffusion Samplingcited by 6
  24. 2026SOAR: Self-Correction for Optimal Alignment and Refinement in Diffusion Modelscited by 6
  25. 2024Conditional Synthesis of 3D Molecules with Time Correction Samplercited by 5
  26. 2025ADT: Tuning Diffusion Models with Adversarial Supervisioncited by 4
  27. 2025Aligning Generative Denoising with Discriminative Objectives Unleashes Diffusion for Visual Perceptioncited by 4
  28. 2025TempoMaster: Efficient Long Video Generation via Next-Frame-Rate Predictioncited by 4
  29. 2025How I Met Your Bias: Investigating Bias Amplification in Diffusion Modelscited by 4
  30. 2026DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Samplingcited by 4
  31. 2026Few-Step Diffusion Sampling Through Instance-Aware Discretizationscited by 4
  32. 2026Continuous-Time Distribution Matching for Few-Step Diffusion Distillationcited by 4
  33. 2026AR Forcing: Towards Long-Horizon Robot Navigation World Modelcited by 4
  34. 2026TempAct: Advancing Temporal Plausibility in Autoregressive Video Generation via Planner-Executor RLcited by 4
  35. 2026Exposure Bias Can Alleviate Itself via Directional and Frequency Rectification in Flow Matchingcited by 4
  36. 2026Explorative Modeling: Unlocking a Third Pretraining Axis and End-to-End Generationcited by 4
  37. 2025Improved Diffusion-based Generative Model with Better Adversarial Robustnesscited by 3
  38. 2025Understanding Flatness in Generative Models: Its Role and Benefitscited by 3

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Unmasking Bias in Diffusion Model Training
  2. 2024Bigger is not Always Better: Scaling Properties of Latent Diffusion Models
  3. 2025Selective Underfitting in Diffusion Models
  4. 2025Deeper Diffusion Models Amplify Bias
  5. 2025Diagnosing and Improving Diffusion Models by Estimating the Optimal Loss Value
  6. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models
  7. 2025Pretrained Diffusion Models Are Inherently Skipped-Step Samplers
  8. 2025Generalization through variance: how noise shapes inductive biases in diffusion models