mlsanityData to 7 Oct 2026

Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models

arXiv 2201.06503 ↗First posted 17 Jan 2022cs.LGcited by 331

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion probabilistic models (DPMs) represent a class of powerful generative models. Despite their success, the inference of DPMs is expensive since it generally needs to iterate over thousands of timesteps. A key problem in the inference is to estimate the variance in each timestep of the reverse process. In this work, we present a surprising result that both the optimal reverse variance and the corresponding optimal KL divergence of a DPM have analytic forms w.r.t. its score function. Building upon it, we propose Analytic-DPM, a training-free inference framework that estimates the analytic forms of the variance and KL divergence using the Monte Carlo method and a pretrained score-based model. Further, to correct the potential bias caused by the score-based model, we derive both lower and upper bounds of the optimal variance and clip the estimate for a better result. Empirically, our analytic-DPM improves the log-likelihood of various DPMs, produces high-quality samples, and meanwhile enjoys a 20x to 80x speed up.

Key facts. Cited by 331 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 37 of its 51 references that are in the map. First posted on arXiv 17 Jan 2022.

Builds on

37 of 51 references.

Closest ancestors · 15

Same direction, newest first

  1. 2021Bilateral Denoising Diffusion Models26 Aug
  2. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  3. 2021Variational Diffusion Models1 Jul
  4. 2021Score-based Generative Modeling in Latent Space10 Jun
  5. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  6. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  7. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  8. 2021UNIT-DDPM: UNpaired Image Translation with Denoising Diffusion Probabilistic Models12 Apr
  9. 2021Noise Estimation for Generative Diffusion Models6 Apr
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  12. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
3 more
  1. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  2. 2020Denoising Diffusion Implicit Models6 Oct
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 22

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  6. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  9. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  11. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  12. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
10 more
  1. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  2. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  3. 2015Importance Weighted Autoencoderscited by 454
  4. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  5. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  6. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  7. 2015A note on the evaluation of generative modelscited by 291
  8. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  9. 2020VAEBM: A Symbiosis between Variational Autoencoders and Energy-based Modelscited by 103
  10. 2019LOGAN: Latent Optimisation for Generative Adversarial Networkscited by 42

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 41; 2023: 125; 2024: 86; 2025: 56; 2026: 23.

331 papers.

Sort:

By area:

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  3. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  7. 2022Diffusion Models in Vision: A Surveycited by 1,008
  8. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  9. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  11. 2023Improving and generalizing flow-based generative models with minibatch optimal transportcited by 835
  12. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
38 more
  1. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  2. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  3. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  4. 2022A Continuous Time Framework for Discrete Denoising Modelscited by 437
  5. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  6. 2024Your Absorbing Discrete Diffusion Secretly Models the Conditional Distributions of Clean Datacited by 371
  7. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  8. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  9. 2023Composer: Creative and Controllable Image Synthesis with Composable Conditionscited by 303
  10. 2023Text-to-3D using Gaussian Splattingcited by 299
  11. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
  12. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  13. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  14. 2022Post-training Quantization on Diffusion Modelscited by 262
  15. 2023LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generationcited by 258
  16. 2022A Survey on Generative Diffusion Modelcited by 238
  17. 2022MM-Diffusion: Learning Multi-Modal Diffusion Models for Joint Audio and Video Generationcited by 225
  18. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  19. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  20. 2022Riemannian Score-Based Generative Modellingcited by 216
  21. 2023Implicit Diffusion Models for Continuous Super-Resolutioncited by 216
  22. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  23. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 207
  24. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motioncited by 186
  25. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 185
  26. 2022Fast Sampling of Diffusion Models via Operator Learningcited by 173
  27. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equationscited by 172
  28. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 167
  29. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165
  30. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 163
  31. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statisticscited by 161
  32. 2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
  33. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 138
  34. 2023A Recipe for Watermarking Diffusion Modelscited by 136
  35. 2023Patch Diffusion: Faster and More Data-Efficient Training of Diffusion Modelscited by 131
  36. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  37. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 127
  38. 2022gDDIM: Generalized denoising diffusion implicit modelscited by 125

50 most cited of 331 All 331 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions
  2. 2024Contractive Diffusion Probabilistic Models
  3. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality
  4. 2025Diffusion Models: A Mathematical Introduction
  5. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions
  6. 2025Assessing the Quality of Denoising Diffusion Models in Wasserstein Distance: Noisy Score and Optimal Bounds
  7. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  8. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps