mlsanityData to 7 Oct 2026

Diffusion Models Meet Contextual Bandits

arXiv 2402.10028 ↗First posted 15 Feb 2024cs.LG cs.AI stat.MLcited by 5

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

Efficient online decision-making in contextual bandits is challenging, as methods without informative priors often suffer from computational or statistical inefficiencies. In this work, we leverage pre-trained diffusion models as expressive priors to capture complex action dependencies and develop a practical algorithm that efficiently approximates posteriors under such priors, enabling both fast updates and sampling. Empirical results demonstrate the effectiveness and versatility of our approach across diverse contextual bandit settings.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 43 of its 76 references that are in the map. First posted on arXiv 15 Feb 2024.

Builds on

43 of 76 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023Thompson Sampling with Diffusion Generative Prior12 Jan
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov
  3. 2022Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning12 Aug

Other references · 40

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  6. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  7. 2013Learning to Optimize Via Posterior Samplingcited by 253
  8. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  9. 2014Online Clustering of Banditscited by 99
  10. 2021Adapting to Misspecification in Contextual Banditscited by 81
  11. 2019Randomized Exploration in Generalized Linear Banditscited by 70
  12. 2021Meta-Thompson Samplingcited by 62
28 more
  1. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  2. 2020Impact of Representation Learning in Linear Banditscited by 51
  3. 2017Multi-Task Learning for Contextual Banditscited by 47
  4. 2020Meta-learning with Stochastic Linear Banditscited by 45
  5. 2021Near-optimal Representation Learning for Linear Bandits and Linear RLcited by 45
  6. 2013Thompson Sampling for Complex Bandit Problemscited by 43
  7. 2020Latent Bandits Revisitedcited by 43
  8. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 43
  9. 2013Thompson Sampling for 1-Dimensional Exponential Family Banditscited by 41
  10. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 41
  11. 2013Sequential Transfer in Multi-armed Bandit with Finite Set of Modelscited by 38
  12. 2020Upper Counterfactual Confidence Bounds: a New Optimism Principle for Contextual Banditscited by 35
  13. 2021Hierarchical Bayesian Banditscited by 33
  14. 2022Contextual Bandits with Large Action Spaces: Made Practicalcited by 32
  15. 2021No Regrets for Learning the Prior in Banditscited by 29
  16. 2018Bayesian Counterfactual Risk Minimizationcited by 28
  17. 2021Metadata-based Multi-Task Bandits with Bayesian Hierarchical Modelscited by 24
  18. 2022Multi-task Representation Learning with Stochastic Linear Banditscited by 20
  19. 2022Lifting the Information Ratio: An Information-Theoretic Analysis of Thompson Sampling for Contextual Banditscited by 19
  20. 2022PAC-Bayesian Offline Contextual Bandits With Guaranteescited by 19
  21. 2022Deep Hierarchy in Banditscited by 18
  22. 2014Bounded Regret for Finite-Armed Structured Banditscited by 15
  23. 2021Metalearning Linear Bandits by Prior Updatecited by 15
  24. 2023Exponential Smoothing for Off-Policy Learningcited by 15
  25. 2022Mixed-Effect Thompson Samplingcited by 14
  26. 2022Towards Scalable and Robust Structured Bandits: A Meta-Learning Frameworkcited by 10
  27. 2023Thompson Sampling Regret Bounds for Contextual Bandits with sub-Gaussian rewardscited by 5
  28. 2023VITS : Variational Inference Thompson Sampling for contextual banditscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 0; 2026: 2.

5 papers.

  1. 2024Bayesian Off-Policy Evaluation and Learning for Large Action Spacescited by 5
  2. 2024Unified PAC-Bayesian Study of Pessimism for Offline Policy Learning with Regularized Importance Samplingcited by 3
  3. 2024Online Posterior Sampling with a Diffusion Priorcited by 2
  4. 2026Bayesian Online Model Selectioncited by 0
  5. 2026SGHA: Evidence-Grounded Research Problem Discovery with Local Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Flow-Corrected Thompson Sampling for Non-Stationary Contextual Bandits
  2. 2023Contextual Bandits and Imitation Learning via Preference-Based Active Queries
  3. 2026RIE-Greedy: Regularization-Induced Exploration for Contextual Bandits
  4. 2019Meta-Learning for Contextual Bandit Exploration
  5. 2021Adapting to Misspecification in Contextual Bandits with Offline Regression Oracles
  6. 2022Flexible and Efficient Contextual Bandits with Heterogeneous Treatment Effect Oracles
  7. 2021GuideBoot: Guided Bootstrap for Deep Contextual Bandits
  8. 2021Offline Neural Contextual Bandits: Pessimism, Optimization and Generalization