# D-Flow: Differentiating through Flows for Controlled Generation

arXiv: https://arxiv.org/abs/2402.14017 · HTML: https://mlsanity.com/paper/2402.14017/

## Key facts

- Cited by 122 papers in the map (data through 14 Oct 2026).
- Placed in Flow matching, time series and science, one of 14 directions in Diffusion models.
- Builds on 32 of its 46 references that are in the map.
- First posted on arXiv 21 Feb 2024.

## Placed in

- Diffusion models / [Flow matching, time series and science](https://mlsanity.com/diffusion/flow-matching-time-series-and-science/)

## Builds on (most cited first, 30 of 32)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [High Fidelity Neural Audio Compression](https://mlsanity.com/paper/2210.13438/) (2022)
- [ILVR: Conditioning Method for Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2108.02938/) (2021)
- [FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Models](https://mlsanity.com/paper/1810.01367/) (2018)
- [Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Model](https://mlsanity.com/paper/2212.00490/) (2022)
- [MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation](https://mlsanity.com/paper/2302.08113/) (2023)
- [Improving Diffusion Models for Inverse Problems using Manifold Constraints](https://mlsanity.com/paper/2206.00941/) (2022)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Compressed Sensing using Generative Models](https://mlsanity.com/paper/1703.03208/) (2017)
- [Cormorant: Covariant Molecular Neural Networks](https://mlsanity.com/paper/1906.04015/) (2019)
- [Fr'echet Audio Distance: A Metric for Evaluating Music Enhancement Algorithms](https://mlsanity.com/paper/1812.08466/) (2018)
- [A Variational Perspective on Solving Inverse Problems with Diffusion Models](https://mlsanity.com/paper/2305.04391/) (2023)
- [Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Models](https://mlsanity.com/paper/2307.00619/) (2023)
- [End-to-End Diffusion Latent Optimization Improves Classifier Guidance](https://mlsanity.com/paper/2303.13703/) (2023)
- [Invertible generative models for inverse problems: mitigating representation error and dataset bias](https://mlsanity.com/paper/1905.11672/) (2019)
- [Generating images of rare concepts using pre-trained diffusion models](https://mlsanity.com/paper/2304.14530/) (2023)
- [Equivariant Flow Matching with Hybrid Probability Transport](https://mlsanity.com/paper/2312.07168/) (2023)
- [Masked Audio Generation using a Single Non-Autoregressive Transformer](https://mlsanity.com/paper/2401.04577/) (2024)
- [On Kinetic Optimal Probability Paths for Generative Models](https://mlsanity.com/paper/2306.06626/) (2023)
- [Solving Inverse Problems with a Flow-based Noise Model](https://mlsanity.com/paper/2003.08089/) (2020)

## Built on by (most cited first, 30 of 122)

- [Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Steps](https://mlsanity.com/paper/2501.09732/) (2025)
- [Adjoint Matching: Fine-tuning Flow and Diffusion Generative Models with Memoryless Stochastic Optimal Control](https://mlsanity.com/paper/2409.08861/) (2024)
- [ReNO: Enhancing One-step Text-to-Image Models through Reward-based Noise Optimization](https://mlsanity.com/paper/2406.04312/) (2024)
- [PnP-Flow: Plug-and-Play Image Restoration with Flow Matching](https://mlsanity.com/paper/2410.02423/) (2024)
- [Not All Noises Are Created Equally:Diffusion Noise Selection and Optimization](https://mlsanity.com/paper/2407.14041/) (2024)
- [Gradient Guidance for Diffusion Models: An Optimization Perspective](https://mlsanity.com/paper/2404.14743/) (2024)
- [Inference-Time Alignment of Diffusion Models with Direct Noise Optimization](https://mlsanity.com/paper/2405.18881/) (2024)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Flow Priors for Linear Inverse Problems via Iterative Corrupted Trajectory Matching](https://mlsanity.com/paper/2405.18816/) (2024)
- [Physics-Constrained Flow Matching: Sampling Generative Models with Hard Constraints](https://mlsanity.com/paper/2506.04171/) (2025)
- [On the Guidance of Flow Matching](https://mlsanity.com/paper/2502.02150/) (2025)
- [Flow Matching with Gaussian Process Priors for Probabilistic Time Series Forecasting](https://mlsanity.com/paper/2410.03024/) (2024)
- [Steering Rectified Flow Models in the Vector Field for Controlled Image Generation](https://mlsanity.com/paper/2412.00100/) (2024)
- [Gradient-Free Generation for Hard-Constrained Systems](https://mlsanity.com/paper/2412.01786/) (2024)
- [Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing](https://mlsanity.com/paper/2503.19385/) (2025)
- [Meta Flow Maps enable scalable reward alignment](https://mlsanity.com/paper/2601.14430/) (2026)
- [Noise Hypernetworks: Amortizing Test-Time Compute in Diffusion Models](https://mlsanity.com/paper/2508.09968/) (2025)
- [Diamond Maps: Efficient Reward Alignment via Stochastic Flow Maps](https://mlsanity.com/paper/2602.05993/) (2026)
- [Physics vs Distributions: Pareto Optimal Flow Matching with Physics Constraints](https://mlsanity.com/paper/2506.08604/) (2025)
- [Energy Matching: Unifying Flow Matching and Energy-Based Models for Generative Modeling](https://mlsanity.com/paper/2504.10612/) (2025)
- [Flower: A Flow-Matching Solver for Inverse Problems](https://mlsanity.com/paper/2509.26287/) (2025)
- [Training Free Guided Flow Matching with Optimal Control](https://mlsanity.com/paper/2410.18070/) (2024)
- [Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Models](https://mlsanity.com/paper/2502.11420/) (2025)
- [Tilt Matching for Scalable Sampling and Fine-Tuning](https://mlsanity.com/paper/2512.21829/) (2025)
- [Diffusion Models as Cartoonists: The Curious Case of High Density Regions](https://mlsanity.com/paper/2411.01293/) (2024)
- [Fast and Robust Visuomotor Riemannian Flow Matching Policy](https://mlsanity.com/paper/2412.10855/) (2024)
- [HOIDiNi: Human-Object Interaction through Diffusion Noise Optimization](https://mlsanity.com/paper/2506.15625/) (2025)
- [GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Models](https://mlsanity.com/paper/2509.25170/) (2025)
- [Free Hunch: Denoiser Covariance Estimation for Diffusion Models Without Extra Costs](https://mlsanity.com/paper/2410.11149/) (2024)
- [Source-Guided Flow Matching](https://mlsanity.com/paper/2508.14807/) (2025)

## Abstract (arXiv)

Taming the generation outcome of state of the art Diffusion and Flow-Matching (FM) models without having to re-train a task-specific model unlocks a powerful tool for solving inverse problems, conditional generation, and controlled generation in general. In this work we introduce D-Flow, a simple framework for controlling the generation process by differentiating through the flow, optimizing for the source (noise) point. We motivate this framework by our key observation stating that for Diffusion/FM models trained with Gaussian probability paths, differentiating through the generation process projects gradient on the data manifold, implicitly injecting the prior into the optimization process. We validate our framework on linear and non-linear controlled generation problems including: image and audio inverse problems and conditional molecule generation reaching state of the art performance across all.