# Regenerative Particle Thompson Sampling

arXiv: https://arxiv.org/abs/2203.08082 · HTML: https://mlsanity.com/paper/2203.08082/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 22 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 4 of 4)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Ensemble Sampling](https://mlsanity.com/paper/1705.07347/) (2017)
- [Thompson Sampling for Complex Bandit Problems](https://mlsanity.com/paper/1311.0466/) (2013)
- [Reward-Biased Maximum Likelihood Estimation for Linear Stochastic Bandits](https://mlsanity.com/paper/2010.04091/) (2020)

## Built on by (most cited first, 1 of 1)

- [Decentralized Multi-Agent Active Search and Tracking when Targets Outnumber Agents](https://mlsanity.com/paper/2401.03154/) (2024)

## Abstract (arXiv)

This paper proposes regenerative particle Thompson sampling (RPTS), a flexible variation of Thompson sampling. Thompson sampling itself is a Bayesian heuristic for solving stochastic bandit problems, but it is hard to implement in practice due to the intractability of maintaining a continuous posterior distribution. Particle Thompson sampling (PTS) is an approximation of Thompson sampling obtained by simply replacing the continuous distribution by a discrete distribution supported at a set of weighted static particles. We observe that in PTS, the weights of all but a few fit particles converge to zero. RPTS is based on the heuristic: delete the decaying unfit particles and regenerate new particles in the vicinity of fit surviving particles. Empirical evidence shows uniform improvement from PTS to RPTS and flexibility and efficacy of RPTS across a set of representative bandit problems, including an application to 5G network slicing.