# Learning to Discretize Denoising Diffusion ODEs

arXiv: https://arxiv.org/abs/2405.15506 · HTML: https://mlsanity.com/paper/2405.15506/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 42 of its 50 references that are in the map.
- First posted on arXiv 24 May 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 42)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [Diffusion Probabilistic Models for 3D Point Cloud Generation](https://mlsanity.com/paper/2103.01458/) (2021)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)

## Built on by (most cited first, 30 of 41)

- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [Accelerated Diffusion Models via Speculative Sampling](https://mlsanity.com/paper/2501.05370/) (2025)
- [Spectral Analysis of Diffusion Models with Application to Schedule Design](https://mlsanity.com/paper/2502.00180/) (2025)
- [Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models](https://mlsanity.com/paper/2507.14797/) (2025)
- [S4S: Solving for a Diffusion Model Solver](https://mlsanity.com/paper/2502.17423/) (2025)
- [Learning to Integrate Diffusion ODEs by Averaging the Derivatives](https://mlsanity.com/paper/2505.14502/) (2025)
- [EVODiff: Entropy-aware Variance Optimized Diffusion Inference](https://mlsanity.com/paper/2509.26096/) (2025)
- [Learnable Sampler Distillation for Discrete Diffusion Models](https://mlsanity.com/paper/2509.19962/) (2025)
- [Optimal Stepsize for Diffusion Sampling](https://mlsanity.com/paper/2503.21774/) (2025)
- [B'ezierFlow: Learning B'ezier Stochastic Interpolant Schedulers for Few-Step Generation](https://mlsanity.com/paper/2512.13255/) (2025)
- [Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling](https://mlsanity.com/paper/2510.23285/) (2025)
- [Rao-Blackwell Gradient Estimators for Equivariant Denoising Diffusion](https://mlsanity.com/paper/2502.09890/) (2025)
- [Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Learnable Linear Extrapolation](https://mlsanity.com/paper/2503.10103/) (2025)
- [Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Models](https://mlsanity.com/paper/2506.10177/) (2025)
- [DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling](https://mlsanity.com/paper/2603.11607/) (2026)
- [Few-Step Diffusion Sampling Through Instance-Aware Discretizations](https://mlsanity.com/paper/2603.17671/) (2026)
- [Parallel Diffusion Solver via Residual Dirichlet Policy Optimization](https://mlsanity.com/paper/2512.22796/) (2025)
- [Mitigating the Contractivity Trap in Diffusion ODEs via Stein Stabilization](https://mlsanity.com/paper/2606.07835/) (2026)
- [Budget-Constrained Step-Level Diffusion Caching](https://mlsanity.com/paper/2606.13496/) (2026)
- [Denoising growth complexity: Data geometry and certified schedules for diffusion sampling](https://mlsanity.com/paper/2607.26285/) (2026)
- [Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations](https://mlsanity.com/paper/2506.20294/) (2025)
- [Analyzing and Guiding Zero-Shot Posterior Sampling in Diffusion Models](https://mlsanity.com/paper/2602.07715/) (2026)
- [Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2603.00763/) (2026)
- [Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction](https://mlsanity.com/paper/2603.03973/) (2026)
- [Three Creates All: You Only Sample 3 Steps](https://mlsanity.com/paper/2603.22375/) (2026)
- [Self-Improving VLA Policies: Selected Diffusion Noise for Spurious-Robust Action Smoothing](https://mlsanity.com/paper/2606.14084/) (2026)
- [DiFA: Inference-Time Forward-Process Alignment for Diffusion Models](https://mlsanity.com/paper/2607.17972/) (2026)
- [Feature Space Analysis by Guided Diffusion Model](https://mlsanity.com/paper/2509.07936/) (2025)
- [GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver](https://mlsanity.com/paper/2510.17699/) (2025)
- [AsyncDiff: Asynchronous Timestep Conditioning for Enhanced Text-to-Image Diffusion Inference](https://mlsanity.com/paper/2512.18675/) (2025)

## Abstract (arXiv)

Diffusion Probabilistic Models (DPMs) are generative models showing competitive performance in various domains, including image synthesis and 3D point cloud generation. Sampling from pre-trained DPMs involves multiple neural function evaluations (NFEs) to transform Gaussian noise samples into images, resulting in higher computational costs compared to single-step generative models such as GANs or VAEs. Therefore, reducing the number of NFEs while preserving generation quality is crucial. To address this, we propose LD3, a lightweight framework designed to learn the optimal time discretization for sampling. LD3 can be combined with various samplers and consistently improves generation quality without having to retrain resource-intensive neural networks. We demonstrate analytically and empirically that LD3 improves sampling efficiency with much less computational overhead. We evaluate our method with extensive experiments on 7 pre-trained models, covering unconditional and conditional sampling in both pixel-space and latent-space DPMs. We achieve FIDs of 2.38 (10 NFE), and 2.27 (10 NFE) on unconditional CIFAR10 and AFHQv2 in 5-10 minutes of training. LD3 offers an efficient approach to sampling from pre-trained diffusion models. Code is available at https://github.com/vinhsuhi/LD3.