Data to 7 Oct 2026

Amortizing intractable inference in diffusion models for vision, language, and control

arXiv 2405.20971 ↗First posted 31 May 2024cs.LG cs.CVcited by 70

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have emerged as effective distribution estimators in vision, language, and reinforcement learning, but their use as priors in downstream tasks poses an intractable posterior inference problem. This paper studies amortized sampling of the posterior over data, x~ prm post(x)propto p(x)r(x), in a model that consists of a diffusion generative model prior p(x) and a black-box constraint or likelihood function r(x). We state and prove the asymptotic correctness of a data-free learning objective, relative trajectory balance, for training a diffusion model that samples from this posterior, a problem that existing methods solve only approximately or in restricted cases. Relative trajectory balance arises from the generative flow network perspective on diffusion models, which allows the use of deep reinforcement learning techniques to improve mode coverage. Experiments illustrate the broad potential of unbiased inference of arbitrary posteriors under diffusion priors: in vision (classifier guidance), language (infilling under a discrete diffusion LLM), and multimodal data (text-to-image generation). Beyond generative modeling, we apply relative trajectory balance to the problem of continuous control with a score-based behavior prior, achieving state-of-the-art results on benchmarks in offline reinforcement learning.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 69 of its 92 references that are in the map. First posted on arXiv 31 May 2024.

Builds on

69 of 92 references.

Closest ancestors · 15

Same direction, newest first

2023Compositional Sculpting of Iterative Generative Processes28 Sep
2023Denoising Diffusion Samplers27 Feb
2023Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC22 Feb
2022Score-based Continuous-time Discrete Diffusion Models30 Nov
2022An optimal control perspective on diffusion-based generative modeling2 Nov
2022Diffusion models as plug-and-play priors17 Jun
2022A Continuous Time Framework for Discrete Denoising Models30 May
2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
2021Diffusion Models Beat GANs on Image Synthesis11 May
2021Improved Denoising Diffusion Probabilistic Models18 Feb
2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan

3 more

2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
2020Denoising Diffusion Implicit Models6 Oct
2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 54

2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
2015Variational Inference with Normalizing Flowscited by 2,161
2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,873
2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375

38 more

2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
2023Training Diffusion Models with Reinforcement Learningcited by 939
2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 750
2022Compositional Visual Generation with Composable Diffusion Modelscited by 619
2022Score Jacobian Chaining: Lifting Pretrained 2D Diffusion Models for 3D Generationcited by 597
2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 504
2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
2018Neural Processescited by 307
2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
2021GFlowNet Foundationscited by 286
2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
2023A Variational Perspective on Solving Inverse Problems with Diffusion Modelscited by 217
2022Continuous diffusion for categorical datacited by 202
2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
2023Likelihood-Based Diffusion Language Modelscited by 147
2023Contrastive Energy Prediction for Exact Energy-Guided Diffusion Sampling in Offline Reinforcement Learningcited by 145
2022Concrete Score Matching: Generalized Score Matching for Discrete Datacited by 144
2024Fine-Tuning of Continuous-Time Diffusion Models as Entropy-Regularized Controlcited by 136
2022Learning GFlowNets from partial episodes for improved convergence and stabilitycited by 129
2023Score-Based Diffusion Models as Principled Priors for Inverse Imagingcited by 126
2024Iterated Denoising Energy Matching for Sampling from Boltzmann Densitiescited by 113
2022GFlowNets and variational inferencecited by 103
2023Improved sampling via learned diffusionscited by 103
2023A theory of continuous generative flow networkscited by 98
2023Amortizing intractable inference in large language modelscited by 94
2020Solving high-dimensional Hamilton-Jacobi-Bellman PDEs using neural networks: perspectives from the theory of controlled diffusions and measures on path spacecited by 80
2020Solving Linear Inverse Problems Using the Prior Implicit in a Denoisercited by 78
2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
2023Diffusion Generative Flow Samplers: Improving learning signals through partial trajectory optimizationcited by 57
2020Consistency of a Recurrent Language Model With Respect to Incomplete Decodingcited by 54

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 16; 2025: 36; 2026: 18.

70 papers.

Sort:

By area:

2024Scaling Diffusion Language Models via Adaptation from Autoregressive Modelscited by 268
2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 127
2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 82
2024Fine-Tuning Discrete Diffusion Models via Reward Optimization with Applications to DNA and Protein Designcited by 72
2024Sequential Controlled Langevin Diffusionscited by 47
2025Diffusion Tree Sampling: Scalable inference-time alignment of diffusion modelscited by 31
2024Efficient Diversity-Preserving Diffusion Alignment via Gradient-Informed GFlowNetscited by 30
2025A Simple and Effective Reinforcement Learning Method for Text-to-Image Diffusion Fine-tuningcited by 30
2025Improving Reasoning for Diffusion Language Models via Group Diffusion Policy Optimizationcited by 30
2025Principled RL for Diffusion LLMs Emerges from a Sequence-Level Perspectivecited by 30
2025Debiasing Guidance for Discrete Diffusion with Sequential Monte Carlocited by 29
2024Steering Masked Discrete Diffusion Models via Discrete Denoising Posterior Predictioncited by 28

38 more

2024Improving GFlowNets for Text-to-Image Diffusion Alignmentcited by 27
2025Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Trainingcited by 27
2024Adaptive teachers for amortized samplerscited by 23
2025Training-Free Guidance Beyond Differentiability: Scalable Path Steering with Tree Search in Diffusion and Flow Modelscited by 23
2025Trust Region Constrained Measure Transport in Path Space for Stochastic Optimal Control and Inferencecited by 22
2024Generative Flows on Synthetic Pathway for Drug Designcited by 20
2025From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster trainingcited by 17
2025Outsourced diffusion sampling: Efficient posterior inference in latent spaces of generative modelscited by 16
2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 16
2025Offline Model-Based Optimization: Comprehensive Reviewcited by 14
2024Improved Off-policy Reinforcement Learning in Biological Sequence Designcited by 13
2024Can a Bayesian Oracle Prevent Harm from an Agent?cited by 12
2024Optimizing Backward Policies in GFlowNets via Trajectory Likelihood Maximizationcited by 12
2025Posterior Inference with Diffusion Models for High-dimensional Black-box Optimizationcited by 12
2025Shaping Inductive Bias in Diffusion Models through Frequency-Based Noise Controlcited by 11
2025Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Designcited by 10
2024Beyond Squared Error: Exploring Loss Design for Enhanced Training of Generative Flow Networkscited by 9
2026Discrete Feynman-Kac Correctorscited by 9
2025Revisiting Non-Acyclic GFlowNets in Discrete Environmentscited by 8
2025Can Knowledge-Graph-based Retrieval Augmented Generation Really Retrieve What You Need?cited by 8
2025Steering Generative Models with Experimental Data for Protein Fitness Optimizationcited by 7
2024Adaptive Diffusion Terrain Generator for Autonomous Uneven Terrain Navigationcited by 6
2025Iterative Importance Fine-tuning of Diffusion Modelscited by 6
2025ReGuidance: A Simple Diffusion Wrapper for Boosting Sample Quality on Hard Inverse Problemscited by 5
2025Reinforced sequential Monte Carlo for amortised samplingcited by 5
2025Diffusion Fine-Tuning via Reparameterized Policy Gradient of the Soft Q-Functioncited by 5
2026Variational Flow Maps: Make Some Noise for One-Step Conditional Generationcited by 5
2024GFlowNet Pretraining with Inexpensive Rewardscited by 4
2024Conditional [MASK] Discrete Diffusion Language Modelcited by 4
2025AbFlowNet: Optimizing Antibody-Antigen Binding Energy via Diffusion-GFlowNet Fusioncited by 4
2025Posterior Inference in Latent Space for Scalable Constrained Black-box Optimizationcited by 4
2025Pretraining Generative Flow Networks with Inexpensive Rewards for Molecular Graph Generationcited by 3
2025Relative Trajectory Balance is equivalent to Trust-PCLcited by 3
2026Supervised Guidance Training for Infinite-Dimensional Diffusion Modelscited by 3
2026f-Trajectory Balance: A Loss Family for Tuning GFlowNets, Generative Models, and LLMs with Off- and On-Policy Datacited by 3
2026Aligning Few-Step Generative Models by Amortizing Sample-based Variational Inferencecited by 3
2026Plug-and-Play Guidance for Discrete Diffusion Models via Gradient-Informed Logit Correctioncited by 3
2025Mitigating Hallucinations in LM-Based TTS Models via Distribution Alignment Using GFlowNetscited by 2

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way