mlsanityData to 7 Oct 2026

Minimax-Bayes Reinforcement Learning

arXiv 2302.10831 ↗First posted 21 Feb 2023cs.LG stat.MLcited by 5

Not on the map. Why?

Abstract (arXiv)

While the Bayesian decision-theoretic framework offers an elegant solution to the problem of decision making under uncertainty, one question is how to appropriately select the prior distribution. One idea is to employ a worst-case prior. However, this is not as easy to specify in sequential decision making as in simple statistical estimation problems. This paper studies (sometimes approximate) minimax-Bayes solutions for various reinforcement learning problems to gain insights into the properties of the corresponding priors and policies. We find that while the worst-case prior depends on the setting, the corresponding minimax policies are more robust than those that assume a standard (i.e. uniform) prior.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 10 of its 31 references that are in the map. First posted on arXiv 21 Feb 2023.

Builds on

10 of 31 references.

References · 10

  1. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  2. 2014Convex Optimization: Algorithms and Complexitycited by 569
  3. 2016Learning to reinforcement learncited by 369
  4. 2019On Gradient Descent Ascent for Nonconvex-Concave Minimax Problemscited by 266
  5. 2021Maximum Entropy RL (Provably) Solves Some Robust RL Problemscited by 169
  6. 2019An Information-Theoretic Approach to Minimax Regret in Partial Monitoringcited by 41
  7. 2019Beyond Confidence Regions: Tight Bayesian Ambiguity Sets for Robust MDPscited by 37
  8. 2020Meta-trained agents implement Bayes-optimal agentscited by 37
  9. 2019A Bayesian Approach to Robust Reinforcement Learningcited by 32
  10. 2014Generalised Entropy MDPs and Minimax Regretcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3.

5 papers.

  1. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  2. 2023Bayesian Exploration Networkscited by 3
  3. 2024Information-Theoretic Minimax Regret Bounds for Reinforcement Learning based on Dualitycited by 1
  4. 2024Isoperimetry is All We Need: Langevin Posterior Sampling for RL with Sublinear Regretcited by 1
  5. 2023Bayesian Neural Networks: A Min-Max Game Frameworkcited by 0