mlsanityData to 14 Oct 2026

Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs

arXiv 2305.03935 ↗First posted 6 May 2023cs.LGcited by 54

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have exhibited excellent performance in various domains. The probability flow ordinary differential equation (ODE) of diffusion models (i.e., diffusion ODEs) is a particular case of continuous normalizing flows (CNFs), which enables deterministic inference and exact likelihood evaluation. However, the likelihood estimation results by diffusion ODEs are still far from those of the state-of-the-art likelihood-based generative models. In this work, we propose several improved techniques for maximum likelihood estimation for diffusion ODEs, including both training and evaluation perspectives. For training, we propose velocity parameterization and explore variance reduction techniques for faster convergence. We also derive an error-bounded high-order flow matching objective for finetuning, which improves the ODE likelihood and smooths its trajectory. For evaluation, we propose a novel training-free truncated-normal dequantization to fill the training-evaluation gap commonly existing in diffusion ODEs. Building upon these techniques, we achieve state-of-the-art likelihood estimation results on image datasets (2.56 on CIFAR-10, 3.43/3.69 on ImageNet-32) without variational dequantization or data augmentation, and 2.42 on CIFAR-10 with data augmentation. Code is available at urlhttps://github.com/thu-ml/i-DODE.

Key facts. Cited by 54 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 50 of its 73 references that are in the map. First posted on arXiv 6 May 2023.

Builds on

50 of 73 references.

Closest ancestors · 14

Same direction, newest first

  1. 2022EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations14 Jul
  2. 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matching16 Jun
  3. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Variational Diffusion Models1 Jul
  7. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimation10 Jun
  8. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  9. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  10. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  11. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  12. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
2 more
  1. 2020Denoising Diffusion Implicit Models6 Oct
  2. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 36

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  9. 2016Density estimation using Real NVPcited by 2,042
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  11. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,545
  12. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
24 more
  1. 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
  2. 2022Denoising Diffusion Restoration Modelscited by 963
  3. 2016Conditional Image Generation with PixelCNN Decoderscited by 862
  4. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  5. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  6. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  7. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 583
  8. 2015Importance Weighted Autoencoderscited by 456
  9. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 417
  10. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  11. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 262
  12. 2020How to train your neural ODE: the world of Jacobian and kinetic regularizationcited by 228
  13. 2019Input complexity and out-of-distribution detection with likelihood-based generative modelscited by 186
  14. 2022Lossy Image Compression with Conditional Diffusion Modelscited by 170
  15. 2021DiffSinger: Singing Voice Synthesis via Shallow Diffusion Mechanismcited by 154
  16. 2020Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encodercited by 145
  17. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  18. 2022Poisson Flow Generative Modelscited by 95
  19. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  20. 2013RNADE: The real-valued neural autoregressive density-estimatorcited by 66
  21. 2021Density Ratio Estimation via Infinitesimal Classificationcited by 61
  22. 2020Lossy Image Compression with Normalizing Flowscited by 25
  23. 2021ANFIC: Image Compression Using Augmented Normalizing Flowscited by 20
  24. 2020Anomaly Detection in Trajectory Data with Normalizing Flowscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 11; 2024: 10; 2025: 24; 2026: 9.

54 papers.

Sort:

By area:

  1. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  2. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
  3. 2024Masked Diffusion Models are Secretly Time-Agnostic Masked Models and Exploit Inaccurate Categorical Samplingcited by 208
  4. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statisticscited by 161
  5. 2023Flow Matching in Latent Spacecited by 160
  6. 2025The Diffusion Dualitycited by 135
  7. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 86
  8. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  9. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  10. 2023Diffusion Models With Learned Adaptive Noisecited by 51
  11. 2024Diffusion Bridge Implicit Modelscited by 49
  12. 2024Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modellingcited by 45
38 more
  1. 2023MissDiff: Training Diffusion Models on Tabular Data with Missing Valuescited by 39
  2. 2023Motion Flow Matching for Human Motion Synthesis and Editingcited by 38
  3. 2025Efficient Diffusion Models: A Surveycited by 35
  4. 2025Continuously Augmented Discrete Diffusion model for Categorical Generative Modelingcited by 34
  5. 2025Does FLUX Already Know How to Perform Physically Plausible Image Composition?cited by 32
  6. 2023Schrodinger Bridges Beat Diffusion Models on Text-to-Speech Synthesiscited by 30
  7. 2024Training Unbiased Diffusion Models From Biased Datasetcited by 25
  8. 2026Causal-rCM: A Unified Teacher-Forcing and Self-Forcing Open Recipe for Autoregressive Diffusion Distillation in Streaming Video Generation and Interactive World Modelscited by 25
  9. 2024Consistency Diffusion Bridge Modelscited by 23
  10. 2025Direct Discriminative Optimization: Your Likelihood-Based Visual Generative Model is Secretly a GAN Discriminatorcited by 23
  11. 2024Diffusion Models as Cartoonists: The Curious Case of High Density Regionscited by 21
  12. 2025Beyond Masked and Unmasked: Discrete Diffusion Models via Partial Maskingcited by 20
  13. 2025Learning normalized image densities via dual score matchingcited by 18
  14. 2023DiffEnc: Variational Diffusion with a Learned Encodercited by 17
  15. 2023Gaussian Interpolation Flowscited by 17
  16. 2025Force Matching with Relativistic Constraints: A Physics-Inspired Approach to Stable and Efficient Generative Modelingcited by 12
  17. 2023InstructPix2NeRF: Instructed 3D Portrait Editing from a Single Imagecited by 10
  18. 2026PosterOmni: Generalized Artistic Poster Creation via Task Distillation and Unified Reward Feedbackcited by 9
  19. 2025FLEX: A Backbone for Diffusion-Based Modeling of Spatio-temporal Physical Systemscited by 8
  20. 2025Rectified Noise: A Generative Model Using Positive-incentive Noisecited by 7
  21. 2024Generating Physical Dynamics under Priorscited by 6
  22. 2025Efficiently Access Diffusion Fisher: Within the Outer Product Span Spacecited by 5
  23. 2025Generative Auto-Bidding in Large-Scale Competitive Auctions via Diffusion Completer-Alignercited by 5
  24. 2025DogFit: Domain-guided Fine-tuning for Efficient Transfer Learning of Diffusion Modelscited by 4
  25. 2025Incomplete Data, Complete Dynamics: A Diffusion Approachcited by 3
  26. 2023Diffusion Models for Interferometric Satellite Aperture Radarcited by 2
  27. 2024Pretrained Reversible Generation as Unsupervised Visual Representation Learningcited by 2
  28. 2025Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Mapscited by 2
  29. 2025Autoregressive regularized score-based diffusion models for multi-scenarios fluid flow predictioncited by 2
  30. 2026Temporal Pair Consistency for Variance-Reduced Flow Matchingcited by 2
  31. 2025Elucidating the Preconditioning in Consistency Distillationcited by 1
  32. 2025Diffusion^2: Dual Diffusion Model with Uncertainty-Aware Adaptive Noise for Momentary Trajectory Predictioncited by 1
  33. 2026When Can Conditional Flow Matching Replace Pointwise Negative Log-Likelihood?cited by 1
  34. 2025Likelihood Matching for Diffusion Modelscited by 0
  35. 2025Score-Based Density Estimation from Pairwise Comparisonscited by 0
  36. 2025Breaking the Likelihood-Quality Trade-off in Diffusion Models by Merging Pretrained Expertscited by 0
  37. 2025FunPhase: A Periodic Functional Autoencoder for Motion Generation via Phase Manifoldscited by 0
  38. 2026Field-Space Autoencoder for Scalable Climate Emulatorscited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025ODEt(ODEl): Shortcutting the Time and the Length in Diffusion and Flow Models for Faster Sampling
  2. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models
  3. 2024An Ordinary Differential Equation Sampler with Stochastic Start for Diffusion Bridge Models
  4. 2025Diffusion Models: A Mathematical Introduction
  5. 2025A-FloPS: Accelerating Diffusion Models via Adaptive Flow Path Sampler
  6. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models
  7. 2021Diffusion Normalizing Flow
  8. 2025TADA: Improved Diffusion Sampling with Training-free Augmented Dynamics