mlsanityData to 7 Oct 2026

Solving Bayesian inverse problems with diffusion priors and off-policy RL

arXiv 2503.09746 ↗First posted 12 Mar 2025cs.LG cs.AI stat.MLcited by 0

On the map: Diffusion models → Inverse problems and image restoration

Abstract (arXiv)

This paper presents a practical application of Relative Trajectory Balance (RTB), a recently introduced off-policy reinforcement learning (RL) objective that can asymptotically solve Bayesian inverse problems optimally. We extend the original work by using RTB to train conditional diffusion model posteriors from pretrained unconditional priors for challenging linear and non-linear inverse problems in vision, and science. We use the objective alongside techniques such as off-policy backtracking exploration to improve training. Importantly, our results show that existing training-free diffusion posterior methods struggle to perform effective posterior inference in latent space due to inherent biases.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Inverse problems and image restoration, one of 14 directions in Diffusion models. Builds on 32 of its 53 references that are in the map. First posted on arXiv 12 Mar 2025.

Builds on

32 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023A Variational Perspective on Solving Inverse Problems with Diffusion Models7 May
  2. 2022Diffusion Posterior Sampling for General Noisy Inverse Problems29 Sep

Other references · 30

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  7. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,665
  8. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  11. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 722
  12. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
18 more
  1. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learningcited by 543
  2. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  3. 2022Trajectory balance: Improved credit assignment in GFlowNetscited by 244
  4. 2022Diffusion models as plug-and-play priorscited by 230
  5. 2022Continuous diffusion for categorical datacited by 202
  6. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  7. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 191
  8. 2021Path Integral Sampler: a stochastic control approach for samplingcited by 175
  9. 2023Efficient Diffusion Policies for Offline Reinforcement Learningcited by 151
  10. 2023Likelihood-Based Diffusion Language Modelscited by 147
  11. 2023A theory of continuous generative flow networkscited by 98
  12. 2023Amortizing intractable inference in large language modelscited by 94
  13. 2020Solving Linear Inverse Problems Using the Prior Implicit in a Denoisercited by 78
  14. 2020VarGrad: A Low-Variance Gradient Estimator for Variational Inferencecited by 73
  15. 2024Amortizing intractable inference in diffusion models for vision, language, and controlcited by 70
  16. 2024Improved off-policy training of diffusion samplerscited by 47
  17. 2019On the well-posedness of Bayesian inverse problemscited by 20
  18. 2022Posterior samples of source galaxies in strong gravitational lenses with score-based priorscited by 20

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2021A Hierarchical Bayesian model for Inverse RL in Partially-Controlled Environments
  2. 2023Bayesian Inverse Transition Learning for Offline Settings
  3. 2026A Stabilized Path-Space Approach to Diffusion-Based Posterior Sampling
  4. 2021Marginal MAP Estimation for Inverse RL under Occlusion with Observer Noise
  5. 2023A Bayesian Approach to Robust Inverse Reinforcement Learning
  6. 2026Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning
  7. 2024Solving General Noisy Inverse Problem via Posterior Sampling: A Policy Gradient Viewpoint
  8. 2021A Generalised Inverse Reinforcement Learning Framework