# Robust and Adaptive Planning under Model Uncertainty

arXiv: https://arxiv.org/abs/1901.02577 · HTML: https://mlsanity.com/paper/1901.02577/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Jan 2019.

## Built on by (most cited first, 11 of 11)

- [Monte Carlo Tree Search: A Review of Recent Modifications and Applications](https://mlsanity.com/paper/2103.04931/) (2021)
- [Risk-Averse Bayes-Adaptive Reinforcement Learning](https://mlsanity.com/paper/2102.05762/) (2021)
- [One Risk to Rule Them All: A Risk-Sensitive Perspective on Model-Based Offline Reinforcement Learning](https://mlsanity.com/paper/2212.00124/) (2022)
- [Robust Policy Learning over Multiple Uncertainty Sets](https://mlsanity.com/paper/2202.07013/) (2022)
- [Two steps to risk sensitivity](https://mlsanity.com/paper/2111.06803/) (2021)
- [Group Distributionally Robust Reinforcement Learning with Hierarchical Latent Variables](https://mlsanity.com/paper/2210.12262/) (2022)
- [Bayesian Risk-Averse Q-Learning with Streaming Observations](https://mlsanity.com/paper/2305.11300/) (2023)
- [Online Robust Planning under Model Uncertainty: A Sample-Based Approach](https://mlsanity.com/paper/2509.10162/) (2025)
- [An Offline Risk-aware Policy Selection Method for Bayesian Markov Decision Processes](https://mlsanity.com/paper/2105.13431/) (2021)
- [Optimizing Minimax Regret in Uncertain MDPs with Small Sets of Policies](https://mlsanity.com/paper/2608.02509/) (2026)
- [Scalable Robust MCTS with Anytime Deterministic Optimality Guarantees](https://mlsanity.com/paper/2610.15461/) (2026)

## Abstract (arXiv)

Planning under model uncertainty is a fundamental problem across many applications of decision making and learning. In this paper, we propose the Robust Adaptive Monte Carlo Planning (RAMCP) algorithm, which allows computation of risk-sensitive Bayes-adaptive policies that optimally trade off exploration, exploitation, and robustness. RAMCP formulates the risk-sensitive planning problem as a two-player zero-sum game, in which an adversary perturbs the agent's belief over the models. We introduce two versions of the RAMCP algorithm. The first, RAMCP-F, converges to an optimal risk-sensitive policy without having to rebuild the search tree as the underlying belief over models is perturbed. The second version, RAMCP-I, improves computational efficiency at the cost of losing theoretical guarantees, but is shown to yield empirical results comparable to RAMCP-F. RAMCP is demonstrated on an n-pull multi-armed bandit problem, as well as a patient treatment scenario.