mlsanityData to 7 Oct 2026

Thompson Sampling with Diffusion Generative Prior

arXiv 2301.05182 ↗First posted 12 Jan 2023cs.LG cs.AI stat.MLcited by 7

On the map: Diffusion models → Diffusion and flow policies for robotics

Abstract (arXiv)

In this work, we initiate the idea of using denoising diffusion models to learn priors for online decision making problems. Our special focus is on the meta-learning for bandit framework, with the goal of learning a strategy that performs well across bandit tasks of a same class. To this end, we train a diffusion model that learns the underlying task distribution and combine Thompson sampling with the learned prior to deal with new tasks at test time. Our posterior sampling algorithm is designed to carefully balance between the learned prior and the noisy observations that come from the learner's interaction with the environment. To capture realistic bandit scenarios, we also propose a novel diffusion model training procedure that trains even from incomplete and/or noisy data, which could be of independent interest. Finally, our extensive experimental evaluations clearly demonstrate the potential of the proposed approach.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Diffusion and flow policies for robotics, one of 14 directions in Diffusion models. Builds on 52 of its 93 references that are in the map. First posted on arXiv 12 Jan 2023.

Builds on

52 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Is Conditional Generative Modeling all you need for Decision-Making?28 Nov

Other references · 51

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  8. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  10. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,419
  11. 2022Diffusion Posterior Sampling for General Noisy Inverse Problemscited by 1,375
  12. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
38 more
  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 2022Denoising Diffusion Restoration Modelscited by 951
  3. 2020Meta-Learning in Neural Networks: A Surveycited by 944
  4. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  5. 2022Improving Diffusion Models for Inverse Problems using Manifold Constraintscited by 501
  6. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptionscited by 440
  7. 2017A Tutorial on Thompson Samplingcited by 420
  8. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  9. 2021Autoregressive Denoising Diffusion Models for Multivariate Probabilistic Time Series Forecastingcited by 332
  10. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Modelscited by 331
  11. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 306
  12. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 284
  13. 2013Learning to Optimize Via Posterior Samplingcited by 253
  14. 2022Diffusion models as plug-and-play priorscited by 230
  15. 2021SNIPS: Solving Noisy Inverse Problems Stochasticallycited by 191
  16. 2022Convergence of score-based generative modeling for general data distributionscited by 179
  17. 2022Fast Sampling of Diffusion Models via Operator Learningcited by 173
  18. 2018Deep Bayesian Bandits Showdown: An Empirical Comparison of Bayesian Deep Networks for Thompson Samplingcited by 166
  19. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  20. 2022Estimating the Optimal Covariance with Imperfect Mean in Diffusion Probabilistic Modelscited by 83
  21. 2018Thompson Sampling for Combinatorial Semi-Banditscited by 78
  22. 2021Meta-Thompson Samplingcited by 62
  23. 2019Information-Theoretic Confidence Bounds for Reinforcement Learningcited by 51
  24. 2022Neural Diffusion Processescited by 50
  25. 2018Unsupervised Learning with Stein's Unbiased Risk Estimatorcited by 47
  26. 2020Meta-learning with Stochastic Linear Banditscited by 45
  27. 2021Stochastic Image Denoising by Sampling from the Posterior Distributioncited by 45
  28. 2020Latent Bandits Revisitedcited by 43
  29. 2021Bayesian decision-making under misspecified priors with applications to meta-learningcited by 43
  30. 2019Meta Dynamic Pricing: Transfer Learning Across Experimentscited by 41
  31. 2021Protein sequence design with deep generative modelscited by 35
  32. 2021No Regrets for Learning the Prior in Banditscited by 29
  33. 2021Meta-Learning Reliable Priors in the Function Spacecited by 23
  34. 2022Conditioning and Sampling in Variational Diffusion Models for Speech Super-Resolutioncited by 20
  35. 2022Deep Hierarchy in Banditscited by 18
  36. 2014Bounded Regret for Finite-Armed Structured Banditscited by 15
  37. 2021Thompson Sampling with a Mixture Priorcited by 15
  38. 2021Metalearning Linear Bandits by Prior Updatecited by 15

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 0; 2026: 1.

7 papers.

  1. 2023GSURE-Based Diffusion Model Training with Corrupted Datacited by 40
  2. 2024Guided Trajectory Generation with Diffusion Models for Offline Model-based Optimizationcited by 15
  3. 2024Diffusion Models Meet Contextual Banditscited by 5
  4. 2024DP-TLDM: Differentially Private Tabular Latent Diffusion Modelcited by 5
  5. 2024Online Posterior Sampling with a Diffusion Priorcited by 2
  6. 2023Online Network Source Optimization with Graph-Kernel MABcited by 1
  7. 2026Bayesian Online Model Selectioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Prior-Guided Diffusion Planning for Offline Reinforcement Learning
  2. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization
  3. 2023Towards Controllable Diffusion Models via Reward-Guided Exploration
  4. 2026Learning To Sample From Diffusion Models Via Inverse Reinforcement Learning
  5. 2023Bridging the Gap: Addressing Discrepancies in Diffusion Model Training for Classifier-Free Guidance
  6. 2023The Choice of Noninformative Priors for Thompson Sampling in Multiparameter Bandit Models
  7. 2020Distilled Thompson Sampling: Practical and Efficient Thompson Sampling via Imitation Learning
  8. 2025Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown