# Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling

arXiv: https://arxiv.org/abs/2404.12940 · HTML: https://mlsanity.com/paper/2404.12940/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 65 of its 84 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 65)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Glow: Generative Flow with Invertible 1x1 Convolutions](https://mlsanity.com/paper/1807.03039/) (2018)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Building Normalizing Flows with Stochastic Interpolants](https://mlsanity.com/paper/2209.15571/) (2022)
- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Improving and generalizing flow-based generative models with minibatch optimal transport](https://mlsanity.com/paper/2302.00482/) (2023)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Score-based Generative Modeling in Latent Space](https://mlsanity.com/paper/2106.05931/) (2021)

## Built on by (most cited first, 30 of 45)

- [Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control](https://mlsanity.com/paper/2409.08861/) (2024)
- [Normalizing Flows are Capable Generative Models](https://mlsanity.com/paper/2412.06329/) (2024)
- [Metric Flow Matching for Smooth Interpolations on the Data Manifold](https://mlsanity.com/paper/2405.14780/) (2024)
- [Fractal Generative Models](https://mlsanity.com/paper/2502.17437/) (2025)
- [Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models](https://mlsanity.com/paper/2508.09968/) (2025)
- [No Trick, No Treat: Pursuits and Challenges Towards Simulation-free Training of Neural Samplers](https://mlsanity.com/paper/2502.06685/) (2025)
- [SDE Matching: Scalable and Simulation-Free Training of Latent Stochastic Differential Equations](https://mlsanity.com/paper/2502.02472/) (2025)
- [Diffusion Models as Cartoonists: The Curious Case of High Density Regions](https://mlsanity.com/paper/2411.01293/) (2024)
- [Curly Flow Matching for Learning Non-gradient Field Dynamics](https://mlsanity.com/paper/2510.26645/) (2025)
- [Learning normalized image densities via dual score matching](https://mlsanity.com/paper/2506.05310/) (2025)
- [FARMER: Flow AutoRegressive Transformer over Pixels](https://mlsanity.com/paper/2510.23588/) (2025)
- [VCT: Training Consistency Models with Variational Noise Coupling](https://mlsanity.com/paper/2502.18197/) (2025)
- [Equivariant Neural Diffusion for Molecule Generation](https://mlsanity.com/paper/2506.10532/) (2025)
- [Straight-Line Diffusion Model for Efficient 3D Molecular Generation](https://mlsanity.com/paper/2503.02918/) (2025)
- [What's the score? Automated Denoising Score Matching for Nonlinear Diffusions](https://mlsanity.com/paper/2407.07998/) (2024)
- [Faster Inference of Flow-Based Generative Models via Improved Data-Noise Coupling](https://mlsanity.com/paper/2603.15279/) (2026)
- [Adaptive Destruction Processes for Diffusion Samplers](https://mlsanity.com/paper/2506.01541/) (2025)
- [Training Latent Diffusion Models with Interacting Particle Algorithms](https://mlsanity.com/paper/2505.12412/) (2025)
- [Cascaded Flow Matching for Heterogeneous Tabular Data with Mixed-Type Features](https://mlsanity.com/paper/2601.22816/) (2026)
- [FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching](https://mlsanity.com/paper/2602.01329/) (2026)
- [FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference](https://mlsanity.com/paper/2602.11105/) (2026)
- [CoVAE: Consistency Training of Variational Autoencoders](https://mlsanity.com/paper/2507.09103/) (2025)
- [Fractional Diffusion Bridge Models](https://mlsanity.com/paper/2511.01795/) (2025)
- [Forward-Learned Discrete Diffusion: Learning how to noise to denoise faster](https://mlsanity.com/paper/2605.18204/) (2026)
- [Learning Straight Flows by Learning Curved Interpolants](https://mlsanity.com/paper/2503.20719/) (2025)
- [Flow Diverse and Efficient: Learning Momentum Flow Matching via Stochastic Velocity Field Sampling](https://mlsanity.com/paper/2506.08796/) (2025)
- [GeodesicNVS: Probability Density Geodesic Flow Matching for Novel View Synthesis](https://mlsanity.com/paper/2603.01010/) (2026)
- [Learning Energy-Based Models from Stochastic Interpolants using Spatiotemporal Differences](https://mlsanity.com/paper/2605.26850/) (2026)
- [On Designing Diffusion Autoencoders for Efficient Generation and Representation Learning](https://mlsanity.com/paper/2506.00136/) (2025)
- [HOTA: Hamiltonian framework for Optimal Transport Advection](https://mlsanity.com/paper/2507.17513/) (2025)

## Abstract (arXiv)

Conventional diffusion models typically relies on a fixed forward process, which implicitly defines complex marginal distributions over latent variables. This can often complicate the reverse process' task in learning generative trajectories, and results in costly inference for diffusion models. To address these limitations, we introduce Neural Flow Diffusion Models (NFDM), a novel framework that enhances diffusion models by supporting a broader range of forward processes beyond the standard Gaussian. We also propose a novel parameterization technique for learning the forward process. Our framework provides an end-to-end, simulation-free optimization objective, effectively minimizing a variational upper bound on the negative log-likelihood. Experimental results demonstrate NFDM's strong performance, evidenced by state-of-the-art likelihood estimation. Furthermore, we investigate NFDM's capacity for learning generative dynamics with specific characteristics, such as deterministic straight lines trajectories, and demonstrate how the framework may be adopted for learning bridges between two distributions. The results underscores NFDM's versatility and its potential for a wide range of applications.