mlsanityData to 14 Oct 2026

Learning to Discretize Denoising Diffusion ODEs

arXiv 2405.15506 ↗First posted 24 May 2024cs.LGcited by 41

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion Probabilistic Models (DPMs) are generative models showing competitive performance in various domains, including image synthesis and 3D point cloud generation. Sampling from pre-trained DPMs involves multiple neural function evaluations (NFEs) to transform Gaussian noise samples into images, resulting in higher computational costs compared to single-step generative models such as GANs or VAEs. Therefore, reducing the number of NFEs while preserving generation quality is crucial. To address this, we propose LD3, a lightweight framework designed to learn the optimal time discretization for sampling. LD3 can be combined with various samplers and consistently improves generation quality without having to retrain resource-intensive neural networks. We demonstrate analytically and empirically that LD3 improves sampling efficiency with much less computational overhead. We evaluate our method with extensive experiments on 7 pre-trained models, covering unconditional and conditional sampling in both pixel-space and latent-space DPMs. We achieve FIDs of 2.38 (10 NFE), and 2.27 (10 NFE) on unconditional CIFAR10 and AFHQv2 in 5-10 minutes of training. LD3 offers an efficient approach to sampling from pre-trained diffusion models. Code is available at https://github.com/vinhsuhi/LD3.

Key facts. Cited by 41 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 42 of its 50 references that are in the map. First posted on arXiv 24 May 2024.

Builds on

42 of 50 references.

Closest ancestors · 30

Same direction, newest first

  1. 2024On the Trajectory Regularity of ODE-based Diffusion Sampling18 May
  2. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Models22 Apr
  3. 2024Accelerating Diffusion Sampling with Optimized Time Steps27 Feb
  4. 2023A Unified Sampling Framework for Solver Searching of Diffusion Probabilistic Models12 Dec
  5. 2023Fast ODE-based Sampling for Diffusion Models in Around 5 Steps30 Nov
  6. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  7. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics20 Oct
  8. 2023Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling4 Aug
  9. 2023Alleviating Exposure Bias in Diffusion Models through Sampling with Shifted Time Steps24 May
  10. 2023Consistency Models2 Mar
  11. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  12. 2023Optimizing DDPM Sampling with Shortcut Fine-Tuning31 Jan
18 more
  1. 2023Input Perturbation Reduces Exposure Bias in Diffusion Models27 Jan
  2. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  3. 2022On Distillation of Guided Diffusion Models6 Oct
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  7. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  8. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  9. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  10. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesis29 Nov
  11. 2021Variational Diffusion Models1 Jul
  12. 2021Learning to Efficiently Sample from Diffusion Probabilistic Models7 Jun
  13. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  14. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  15. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  16. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  17. 2020Denoising Diffusion Implicit Models6 Oct
  18. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 12

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  7. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,355
  8. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,098
  9. 2021Diffusion Probabilistic Models for 3D Point Cloud Generationcited by 735
  10. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  11. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 395
  12. 2022DiffDock: Diffusion Steps, Twists, and Turns for Molecular Dockingcited by 313

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 19; 2026: 22.

41 papers.

Sort:

By area:

  1. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  2. 2025Accelerated Diffusion Models via Speculative Samplingcited by 31
  3. 2025Spectral Analysis of Diffusion Models with Application to Schedule Designcited by 13
  4. 2025Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Modelscited by 13
  5. 2025S4S: Solving for a Diffusion Model Solvercited by 10
  6. 2025Learning to Integrate Diffusion ODEs by Averaging the Derivativescited by 10
  7. 2025EVODiff: Entropy-aware Variance Optimized Diffusion Inferencecited by 10
  8. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  9. 2025Optimal Stepsize for Diffusion Samplingcited by 7
  10. 2025B'ezierFlow: Learning B'ezier Stochastic Interpolant Schedulers for Few-Step Generationcited by 7
  11. 2025Adaptive Stochastic Coefficients for Accelerating Diffusion Samplingcited by 6
  12. 2025Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusioncited by 5
29 more
  1. 2025Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Learnable Linear Extrapolationcited by 4
  2. 2025Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Modelscited by 4
  3. 2026DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Samplingcited by 4
  4. 2026Few-Step Diffusion Sampling Through Instance-Aware Discretizationscited by 4
  5. 2025Parallel Diffusion Solver via Residual Dirichlet Policy Optimizationcited by 3
  6. 2026Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilizationcited by 3
  7. 2026Budget-Constrained Step-Level Diffusion Cachingcited by 3
  8. 2026Denoising growth complexity: Data geometry and certified schedules for diffusion samplingcited by 2
  9. 2025Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorationscited by 1
  10. 2026Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Modelscited by 1
  11. 2026Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Modelscited by 1
  12. 2026Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Predictioncited by 1
  13. 2026Three Creates All: You Only Sample 3 Stepscited by 1
  14. 2026Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothingcited by 1
  15. 2026DiFA: Inference-Time Forward-Process Alignment for Diffusion Modelscited by 1
  16. 2025Feature Space Analysis by Guided Diffusion Modelcited by 0
  17. 2025GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solvercited by 0
  18. 2025AsyncDiff: Asynchronous Timestep Conditioning for Enhanced Text-to-Image Diffusion Inferencecited by 0
  19. 2026A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Modelscited by 0
  20. 2026Information-geometric adaptive sampling for graph diffusioncited by 0
  21. 2026SymDrift: One-Shot Generative Modeling under Symmetriescited by 0
  22. 2026Stability and Discretization Error of State Space Model Neural Operatorscited by 0
  23. 2026Learning to Solve Generative ODEs Beyond the Linear Spancited by 0
  24. 2026ARIA: Adaptive Region-Based Importance Allocation for Conditional Diffusion Distillationcited by 0
  25. 2026D2PO: Optimizing Diffusion Samplers via Dynamic Preferencecited by 0
  26. 2026RAIN: Region-Aware Inversion Network for Semantic Watermark Extractioncited by 0
  27. 2026I-SplineFlow: Learning Monotone Spline Stochastic Interpolant Schedulers for Few-Step Generationcited by 0
  28. 2026Unlocking Few-Step Diffusion for Faithful Previewscited by 0
  29. 2026A Step in the Right Place: Understanding and Optimizing Timestep Placement in Continuous Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis
  2. 2024Denoising Diffusion Probabilistic Models in Six Simple Steps
  3. 2024Optimizing Few-Step Sampler for Diffusion Probabilistic Model
  4. 2025Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approach
  5. 2023The Missing U for Efficient Diffusion Models
  6. 2025Enhanced Diffusion Sampling via Extrapolation with Multiple ODE Solutions
  7. 2024PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores
  8. 2024Diffusion Bridge Implicit Models