mlsanityData to 7 Oct 2026

Pseudo Numerical Methods for Diffusion Models on Manifolds

arXiv 2202.09778 ↗First posted 20 Feb 2022cs.CV cs.LG cs.NAcited by 695

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Denoising Diffusion Probabilistic Models (DDPMs) can generate high-quality samples such as image and audio samples. However, DDPMs require hundreds to thousands of iterations to produce final samples. Several prior works have successfully accelerated DDPMs through adjusting the variance schedule (e.g., Improved Denoising Diffusion Probabilistic Models) or the denoising equation (e.g., Denoising Diffusion Implicit Models (DDIMs)). However, these acceleration methods cannot maintain the quality of samples and even introduce new noise at a high speedup rate, which limit their practicability. To accelerate the inference process while keeping the sample quality, we provide a fresh perspective that DDPMs should be treated as solving differential equations on manifolds. Under such a perspective, we propose pseudo numerical methods for diffusion models (PNDMs). Specifically, we figure out how to solve differential equations on manifolds and show that DDIMs are simple cases of pseudo numerical methods. We change several classical numerical methods to corresponding pseudo numerical methods and find that the pseudo linear multi-step method is the best in most situations. According to our experiments, by directly using pre-trained models on Cifar10, CelebA and LSUN, PNDMs can generate higher quality synthetic images with only 50 steps compared with 1000-step DDIMs (20x speedup), significantly outperform DDIMs with 250 steps (by around 0.4 in FID) and have good generalization on different variance schedules. Our implementation is available at https://github.com/luping-liu/PNDM.

Key facts. Cited by 695 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 23 of its 35 references that are in the map. First posted on arXiv 20 Feb 2022.

Builds on

23 of 35 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021Bilateral Denoising Diffusion Models26 Aug
  2. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models6 Aug
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  4. 2021Score-based Generative Modeling in Latent Space10 Jun
  5. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  6. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  7. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Implicit Models6 Oct
  10. 2020Denoising Diffusion Probabilistic Models19 Jun
  11. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 12

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2018Neural Ordinary Differential Equationscited by 3,733
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  6. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  7. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  8. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 734
  9. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  10. 2019Augmented Neural ODEscited by 396
  11. 2021Diff-TTS: A Denoising Diffusion Model for Text-to-Speechcited by 156
  12. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 65; 2023: 208; 2024: 199; 2025: 153; 2026: 70.

695 papers.

Sort:

By area:

  1. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  6. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  7. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  8. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  9. 2022Diffusion Models in Vision: A Surveycited by 1,008
  10. 2022Token Merging: Your ViT But Fastercited by 995
  11. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  12. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
38 more
  1. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  2. 2022On Distillation of Guided Diffusion Modelscited by 721
  3. 2023ModelScope Text-to-Video Technical Reportcited by 651
  4. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  5. 2022LION: Latent Point Diffusion Models for 3D Shape Generationcited by 552
  6. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 529
  7. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  8. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  9. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 441
  10. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  11. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  12. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 386
  13. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  14. 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
  15. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  16. 2023I2VGen-XL: High-Quality Image-to-Video Synthesis via Cascaded Diffusion Modelscited by 343
  17. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  18. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  19. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  20. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  21. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  22. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 284
  23. 2023Preserve Your Own Correlation: A Noise Prior for Video Diffusion Modelscited by 284
  24. 2022SmartBrush: Text and Shape Guided Object Inpainting with Diffusion Modelcited by 276
  25. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 266
  26. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 266
  27. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  28. 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 256
  29. 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
  30. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 242
  31. 2022A Survey on Generative Diffusion Modelcited by 238
  32. 2022What the DAAM: Interpreting Stable Diffusion Using Cross Attentioncited by 233
  33. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  34. 2023Token Merging for Fast Stable Diffusioncited by 220
  35. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  36. 2023DIFUSCO: Graph-based Diffusion Solvers for Combinatorial Optimizationcited by 216
  37. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  38. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 211

50 most cited of 695 All 695 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  2. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
  3. 2024Diffusion Bridge Implicit Models
  4. 2023UDPM: Upsampling Diffusion Probabilistic Models
  5. 2022Convergence of denoising diffusion models under the manifold hypothesis
  6. 2025Riemannian Denoising Diffusion Probabilistic Models
  7. 2024Score-Optimal Diffusion Schedules
  8. 2023Metropolis Sampling for Constrained Diffusion Models