# Minimax-Bayes Reinforcement Learning

arXiv: https://arxiv.org/abs/2302.10831 · HTML: https://mlsanity.com/paper/2302.10831/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 31 references that are in the map.
- First posted on arXiv 21 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Learning to reinforcement learn](https://mlsanity.com/paper/1611.05763/) (2016)
- [On Gradient Descent Ascent for Nonconvex-Concave Minimax Problems](https://mlsanity.com/paper/1906.00331/) (2019)
- [Maximum Entropy RL (Provably) Solves Some Robust RL Problems](https://mlsanity.com/paper/2103.06257/) (2021)
- [An Information-Theoretic Approach to Minimax Regret in Partial Monitoring](https://mlsanity.com/paper/1902.00470/) (2019)
- [Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPs](https://mlsanity.com/paper/1902.07605/) (2019)
- [Meta-trained agents implement Bayes-optimal agents](https://mlsanity.com/paper/2010.11223/) (2020)
- [A Bayesian Approach to Robust Reinforcement Learning](https://mlsanity.com/paper/1905.08188/) (2019)
- [Generalised Entropy MDPs and Minimax Regret](https://mlsanity.com/paper/1412.3276/) (2014)

## Built on by (most cited first, 6 of 6)

- [MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximization](https://mlsanity.com/paper/2412.12098/) (2024)
- [Bayesian Exploration Networks](https://mlsanity.com/paper/2308.13049/) (2023)
- [Information-Theoretic Minimax Regret Bounds for Reinforcement Learning based on Duality](https://mlsanity.com/paper/2410.16013/) (2024)
- [Isoperimetry is All We Need: Langevin Posterior Sampling for RL with Sublinear Regret](https://mlsanity.com/paper/2412.20824/) (2024)
- [Bayesian Neural Networks: A Min-Max Game Framework](https://mlsanity.com/paper/2311.11126/) (2023)
- [Optimizing Social Utility in Sequential Experiments](https://mlsanity.com/paper/2605.06520/) (2026)

## Abstract (arXiv)

While the Bayesian decision-theoretic framework offers an elegant solution to the problem of decision making under uncertainty, one question is how to appropriately select the prior distribution. One idea is to employ a worst-case prior. However, this is not as easy to specify in sequential decision making as in simple statistical estimation problems. This paper studies (sometimes approximate) minimax-Bayes solutions for various reinforcement learning problems to gain insights into the properties of the corresponding priors and policies. We find that while the worst-case prior depends on the setting, the corresponding minimax policies are more robust than those that assume a standard (i.e. uniform) prior.