mlsanityData to 7 Oct 2026

Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation

arXiv 2303.00848 ↗First posted 1 Mar 2023cs.LG cs.AI stat.MLcited by 293

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

To achieve the highest perceptual quality, state-of-the-art diffusion models are optimized with objectives that typically look very different from the maximum likelihood and the Evidence Lower Bound (ELBO) objectives. In this work, we reveal that diffusion model objectives are actually closely related to the ELBO. Specifically, we show that all commonly used diffusion model objectives equate to a weighted integral of ELBOs over different noise levels, where the weighting depends on the specific objective used. Under the condition of monotonic weighting, the connection is even closer: the diffusion objective then equals the ELBO, combined with simple data augmentation, namely Gaussian noise perturbation. We show that this condition holds for a number of state-of-the-art diffusion models. In experiments, we explore new monotonic weightings and demonstrate their effectiveness, achieving state-of-the-art FID scores on the high-resolution ImageNet benchmark.

Key facts. Cited by 293 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 41 of its 54 references that are in the map. First posted on arXiv 1 Mar 2023.

Builds on

41 of 54 references.

Closest ancestors · 16

Same direction, newest first

  1. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2022All are Worth Words: A ViT Backbone for Diffusion Models25 Sep
  4. 2022Classifier-Free Diffusion Guidance26 Jul
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2022Perception Prioritized Training of Diffusion Models1 Apr
  7. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  8. 2021Variational Diffusion Models1 Jul
  9. 2021Score-based Generative Modeling in Latent Space10 Jun
  10. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  11. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  12. 2021Diffusion Models Beat GANs on Image Synthesis11 May
4 more
  1. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  2. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 25

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2022Flow Matching for Generative Modelingcited by 5,426
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2016Improved Techniques for Training GANscited by 2,938
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  11. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  12. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
13 more
  1. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  4. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  5. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  6. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  7. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  8. 2022Novel View Synthesis with Diffusion Modelscited by 299
  9. 2021Deblurring via Stochastic Refinementcited by 248
  10. 2022Continuous diffusion for categorical datacited by 202
  11. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  12. 2021WaveGrad 2: Iterative Refinement for Text-to-Speech Synthesiscited by 65
  13. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 80; 2025: 115; 2026: 91.

293 papers.

Sort:

By area:

  1. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  2. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  3. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  4. 2024Autoregressive Image Generation without Vector Quantizationcited by 644
  5. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  6. 2024One Step Diffusion via Shortcut Modelscited by 373
  7. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 353
  8. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  9. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  10. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 159
  11. 2024Consistency Models Made Easycited by 140
  12. 2025Training Agents Inside of Scalable World Modelscited by 122
38 more
  1. 2025Inductive Moment Matchingcited by 107
  2. 2024Rolling Diffusion Modelscited by 102
  3. 2026Generative Modeling via Driftingcited by 98
  4. 2025Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Thinkcited by 95
  5. 2025PixelFlow: Pixel-Space Generative Models with Flowcited by 91
  6. 2024Large Concept Models: Language Modeling in a Sentence Representation Spacecited by 86
  7. 2025Flow Matching Policy Gradientscited by 86
  8. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  9. 2024Variational Flow Matching for Graph Generationcited by 78
  10. 2025PixNerd: Pixel Neural Field Diffusioncited by 75
  11. 2025PixelDiT: Pixel Diffusion Transformers for Image Generationcited by 75
  12. 2024Discrete-state Continuous-time Diffusion for Graph Generationcited by 73
  13. 2025Generalized Interpolating Discrete Diffusioncited by 68
  14. 2024Scaling Diffusion Transformers to 16 Billion Parameterscited by 65
  15. 2024FasterDiT: Towards Faster Diffusion Transformers Training without Architecture Modificationcited by 64
  16. 2025Proteina: Scaling Flow-based Protein Structure Generative Modelscited by 63
  17. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  18. 2024EM Distillation for One-step Diffusion Modelscited by 60
  19. 2025Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Modelscited by 59
  20. 2024SoftVQ-VAE: Efficient 1-Dimensional Continuous Tokenizercited by 55
  21. 2025DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generationcited by 55
  22. 2023Navigating the Design Space of Equivariant Diffusion-Based Generative Models for De Novo 3D Molecule Generationcited by 50
  23. 2024Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matchingcited by 50
  24. 2025No Other Representation Component Is Needed: Diffusion Transformers Can Provide Representation Guidance by Themselvescited by 50
  25. 2025DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generationcited by 49
  26. 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 49
  27. 2025A Fourier Space Perspective on Diffusion Modelscited by 48
  28. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  29. 2024Adversarial Score identity Distillation: Rapidly Surpassing the Teacher in One Stepcited by 47
  30. 2024Scalable Diffusion Models with State Space Backbonecited by 45
  31. 2024OmniFlow: Any-to-Any Generation with Multi-Modal Rectified Flowscited by 45
  32. 2024JetFormer: An Autoregressive Generative Model of Raw Images and Textcited by 44
  33. 2025Fractal Generative Modelscited by 44
  34. 2024Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modellingcited by 43
  35. 2025DiP: Taming Diffusion Models in Pixel Spacecited by 42
  36. 2024Diffusion-RWKV: Scaling RWKV-Like Architectures for Diffusion Modelscited by 41
  37. 2024Diffusion Policies creating a Trust Region for Offline Reinforcement Learningcited by 41
  38. 2025Calibrated Multi-Preference Optimization for Aligning Diffusion Modelscited by 40

50 most cited of 293 All 293 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion Models With Learned Adaptive Noise
  2. 2023Unmasking Bias in Diffusion Model Training
  3. 2026Revisiting Classifier-Free Guidance Methods in Latent Diffusion Models
  4. 2023Diffusion Model with Perceptual Loss
  5. 2023DiffEnc: Variational Diffusion with a Learned Encoder
  6. 2023Analyzing and Improving the Training Dynamics of Diffusion Models
  7. 2024Simpler Diffusion (SiD2): 1.5 FID on ImageNet512 with pixel-space diffusion
  8. 2025Diffusion Models without Classifier-free Guidance