mlsanityData to 14 Oct 2026

Online Posterior Sampling with a Diffusion Prior

arXiv 2410.03919 ↗First posted 4 Oct 2024cs.LG stat.MLcited by 2

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Posterior sampling in contextual bandits with a Gaussian prior can be implemented exactly or approximately using the Laplace approximation. The Gaussian prior is computationally efficient but it cannot describe complex distributions. In this work, we propose approximate posterior sampling algorithms for contextual bandits with a diffusion model prior. The key idea is to sample from a chain of approximate conditional posteriors, one for each stage of the reverse diffusion process, which are obtained by the Laplace approximation. Our approximations are motivated by posterior sampling with a Gaussian prior, and inherit its simplicity and efficiency. They are asymptotically consistent and perform well empirically on a variety of contextual bandit problems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 29 of its 53 references that are in the map. First posted on arXiv 4 Oct 2024.

Builds on

29 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Diffusion Models Meet Contextual Bandits15 Feb
  2. 2023Thompson Sampling with Diffusion Generative Prior12 Jan

Other references · 27

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,416
  6. 2016Hyperband: A Novel Bandit-Based Approach to Hyperparameter Optimizationcited by 699
  7. 2022Zero-Shot Image Restoration Using Denoising Diffusion Null-Space Modelcited by 583
  8. 2023Universal Guidance for Diffusion Modelscited by 437
  9. 2017A Tutorial on Thompson Samplingcited by 433
  10. 2013Learning to Optimize Via Posterior Samplingcited by 256
  11. 2023Practical and Asymptotically Exact Conditional Sampling in Diffusion Modelscited by 232
  12. 2022Diffusion models as plug-and-play priorscited by 231
15 more
  1. 2023Solving Inverse Problems with Latent Diffusion Models via Hard Data Consistencycited by 182
  2. 2014An Information-Theoretic Analysis of Thompson Samplingcited by 178
  3. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 169
  4. 2023Solving Linear Inverse Problems Provably via Posterior Sampling with Latent Diffusion Modelscited by 157
  5. 2020Parametric UMAP embeddings for representation and semi-supervised learningcited by 104
  6. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Modelscited by 83
  7. 2019Randomized Exploration in Generalized Linear Banditscited by 70
  8. 2017Scalable Generalized Linear Bandits: Online Computation and Hashingcited by 64
  9. 2022Diffusion Model Based Posterior Sampling for Noisy Linear Inverse Problemscited by 60
  10. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  11. 2021Hierarchical Bayesian Banditscited by 35
  12. 2021No Regrets for Learning the Prior in Banditscited by 30
  13. 2022Deep Hierarchy in Banditscited by 19
  14. 2021Thompson Sampling with a Mixture Priorcited by 16
  15. 2022Mixed-Effect Thompson Samplingcited by 14

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Leveraging priors on distribution functions for multi-arm banditscited by 0
  2. 2025In-Context Decision Making for Optimizing Complex AutoML Pipelinescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
  2. 2024Thompson Sampling for Stochastic Bandits with Noisy Contexts: An Information-Theoretic Regret Analysis
  3. 2025Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown
  4. 2023Thompson Sampling for Linear Bandit Problems with Normal-Gamma Priors
  5. 2025Provable Anytime Ensemble Sampling Algorithms in Nonlinear Contextual Bandits
  6. 2020Tractable contextual bandits beyond realizability
  7. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles
  8. 2025Q-learning with Posterior Sampling