# Thompson Sampling for Parameterized Markov Decision Processes with Uninformative Actions

arXiv: https://arxiv.org/abs/2305.07844 · HTML: https://mlsanity.com/paper/2305.07844/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 29 references that are in the map.
- First posted on arXiv 13 May 2023.

## Builds on (most cited first, 3 of 3)

- [A Tutorial on Thompson Sampling](https://mlsanity.com/paper/1707.02038/) (2017)
- [Learning to Optimize Via Posterior Sampling](https://mlsanity.com/paper/1301.2609/) (2013)
- [(More) Efficient Reinforcement Learning via Posterior Sampling](https://mlsanity.com/paper/1306.0940/) (2013)

## Built on by (most cited first, 1 of 1)

- [Thompson Sampling for Infinite-Horizon Discounted Decision Processes](https://mlsanity.com/paper/2405.08253/) (2024)

## Abstract (arXiv)

We study parameterized MDPs (PMDPs) in which the key parameters of interest are unknown and must be learned using Bayesian inference. One key defining feature of such models is the presence of "uninformative" actions that provide no information about the unknown parameters. We contribute a set of assumptions for PMDPs under which Thompson sampling guarantees an asymptotically optimal expected regret bound of O(T-1), which are easily verified for many classes of problems such as queuing, inventory control, and dynamic pricing.