mlsanityData to 7 Oct 2026

Diffusion Normalizing Flow

arXiv 2110.07579 ↗First posted 14 Oct 2021cs.LGcited by 77

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We present a novel generative modeling method called diffusion normalizing flow based on stochastic differential equations (SDEs). The algorithm consists of two neural SDEs: a forward SDE that gradually adds noise to the data to transform the data into Gaussian random noise, and a backward SDE that gradually removes the noise to sample from the data distribution. By jointly training the two neural SDEs to minimize a common cost function that quantifies the difference between the two, the backward SDE converges to a diffusion process the starts with a Gaussian distribution and ends with the desired data distribution. Our method is closely related to normalizing flow and diffusion probabilistic models and can be viewed as a combination of the two. Compared with normalizing flow, diffusion normalizing flow is able to learn distributions with sharp boundaries. Compared with diffusion probabilistic models, diffusion normalizing flow requires fewer discretization steps and thus has better sampling efficiency. Our algorithm demonstrates competitive performance in both high-dimension data density estimation and image generation tasks.

Key facts. Cited by 77 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 30 of its 41 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

30 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Probabilistic Models19 Jun
  4. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 26

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2018Neural Ordinary Differential Equationscited by 3,733
  6. 2015Variational Inference with Normalizing Flowscited by 2,161
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  9. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  11. 2017Masked Autoregressive Flow for Density Estimationcited by 680
  12. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 584
14 more
  1. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  2. 2015MADE: Masked Autoencoder for Distribution Estimationcited by 306
  3. 2020Scalable Gradients for Stochastic Differential Equationscited by 264
  4. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  5. 2019Residual Flows for Invertible Generative Modelingcited by 256
  6. 2020Learning Gradient Fields for Shape Generationcited by 245
  7. 2018Neural Autoregressive Flowscited by 207
  8. 2020Stochastic Normalizing Flowscited by 156
  9. 2020Residual Energy-Based Models for Text Generationcited by 95
  10. 2020Adversarial score matching and improved sampling for image generationcited by 95
  11. 2018Transformation Autoregressive Networkscited by 27
  12. 2019A RAD approach to deep mixture modelscited by 18
  13. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18
  14. 2017Learning to Generate Samples from Noise through Infusion Trainingcited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 12; 2023: 21; 2024: 19; 2025: 16; 2026: 5.

77 papers.

Sort:

By area:

  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  2. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  3. 2022Diffusion Models in Vision: A Surveycited by 1,008
  4. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  5. 2022A Survey on Generative Diffusion Modelcited by 238
  6. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 221
  7. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  8. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  9. 2022gDDIM: Generalized denoising diffusion implicit modelscited by 125
  10. 2023Minimizing Trajectory Curvature of ODE-based Generative Modelscited by 87
  11. 2024Diffusion Models in Low-Level Vision: A Surveycited by 73
  12. 2022Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidancecited by 67
38 more
  1. 2024Diffusion Models, Image Super-Resolution And Everything: A Surveycited by 63
  2. 2021Bayesian Learning via Neural Schr"odinger-F"ollmer Flowscited by 56
  3. 2022Efficient Diffusion Models for Vision: A Surveycited by 50
  4. 2023Diffusion Models With Learned Adaptive Noisecited by 50
  5. 2024Improved off-policy training of diffusion samplerscited by 47
  6. 2024Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modellingcited by 43
  7. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  8. 2022Normalizing flow neural networks by JKO schemecited by 39
  9. 2025Efficient Diffusion Models: A Surveycited by 35
  10. 2023A mean-field games laboratory for generative modelingcited by 28
  11. 2021Generalized Normalizing Flows via Markov Chainscited by 23
  12. 2023DiffMotion: Speech-Driven Gesture Synthesis Using Denoising Diffusion Modelcited by 22
  13. 2024Mutual Information Estimation via Normalizing Flowscited by 22
  14. 2023Transport meets Variational Inference: Controlled Monte Carlo Diffusionscited by 21
  15. 2025Convergence of Deterministic and Stochastic Diffusion-Model Samplers: A Simple Analysis in Wasserstein Distancecited by 20
  16. 2024Diffusion Models as Cartoonists: The Curious Case of High Density Regionscited by 19
  17. 2023Gaussian Mixture Solvers for Diffusion Modelscited by 17
  18. 2023Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Samplingcited by 16
  19. 2025GenPO: Generative Diffusion Models Meet On-Policy Reinforcement Learningcited by 16
  20. 2023Solving Inverse Physics Problems with Score Matchingcited by 15
  21. 2023Diffusion Models in Bioinformatics: A New Wave of Deep Learning Revolution in Actioncited by 15
  22. 2023Star-Shaped Denoising Diffusion Probabilistic Modelscited by 13
  23. 2022Dimensionality-Varying Diffusion Processcited by 12
  24. 2024Structure-Guided Adversarial Training of Diffusion Modelscited by 11
  25. 2022A Flexible Diffusion Modelcited by 10
  26. 2023Accelerating Markov Chain Monte Carlo sampling with diffusion modelscited by 10
  27. 2024Generative AI and Process Systems Engineering: The Next Frontiercited by 8
  28. 2023Diffusion Models with Deterministic Normalizing Flow Priorscited by 7
  29. 2023Elucidating the solution space of extended reverse-time SDE for diffusion modelscited by 7
  30. 2023Bayesian Gaussian Process ODEs via Double Normalizing Flowscited by 7
  31. 2024Improving Neural Optimal Transport via Displacement Interpolationcited by 7
  32. 2025Global Search for Optimal Low Thrust Spacecraft Trajectories using Diffusion Models and the Indirect Methodcited by 7
  33. 2023Restoration based Generative Modelscited by 6
  34. 2024Diffusion Bridge AutoEncoders for Unsupervised Representation Learningcited by 6
  35. 2025MatSwap: Light-aware material transfers in imagescited by 6
  36. 2025Diffusion-Sharpening: Fine-tuning Diffusion Models with Denoising Trajectory Sharpeningcited by 6
  37. 2024Your Diffusion Model is Secretly a Noise Classifier and Benefits from Contrastive Trainingcited by 5
  38. 2024Learning Efficient and Effective Trajectories for Differential Equation-based Image Restorationcited by 5

50 most cited of 77 All 77 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Non-Normal Diffusion Models
  2. 2022Flow Matching for Generative Modeling
  3. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs
  4. 2025An Introduction to Flow Matching and Diffusion Models
  5. 2023Exploring the Optimal Choice for Generative Processes in Diffusion Models: Ordinary vs Stochastic Differential Equations
  6. 2022Building Normalizing Flows with Stochastic Interpolants
  7. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  8. 2025Diffusion Models: A Mathematical Introduction