# Constrained Hierarchical Monte Carlo Belief-State Planning

arXiv: https://arxiv.org/abs/2310.20054 · HTML: https://mlsanity.com/paper/2310.20054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 32 references that are in the map.
- First posted on arXiv 30 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Code as Policies: Language Model Programs for Embodied Control](https://mlsanity.com/paper/2209.07753/) (2022)
- [Large Language Models as Commonsense Knowledge for Large-Scale Task Planning](https://mlsanity.com/paper/2305.14078/) (2023)
- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [Bayesian Optimized Monte Carlo Planning](https://mlsanity.com/paper/2010.03597/) (2020)
- [BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximations](https://mlsanity.com/paper/2306.00249/) (2023)
- [Closing the Planning-Learning Loop with Application to Autonomous Driving](https://mlsanity.com/paper/2101.03834/) (2021)
- [PLGRIM: Hierarchical Value Learning for Large-scale Exploration in Unknown Environments](https://mlsanity.com/paper/2102.05633/) (2021)
- [A POMDP Model for Safe Geological Carbon Sequestration](https://mlsanity.com/paper/2212.00669/) (2022)
- [C-MCTS: Safe Planning with Monte Carlo Tree Search](https://mlsanity.com/paper/2305.16209/) (2023)
- [Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent](https://mlsanity.com/paper/2212.12154/) (2022)

## Built on by (most cited first, 1 of 1)

- [Anytime Probabilistically Constrained Provably Convergent Online Belief Space Planning](https://mlsanity.com/paper/2411.06711/) (2024)

## Abstract (arXiv)

Optimal plans in Constrained Partially Observable Markov Decision Processes (CPOMDPs) maximize reward objectives while satisfying hard cost constraints, generalizing safe planning under state and transition uncertainty. Unfortunately, online CPOMDP planning is extremely difficult in large or continuous problem domains. In many large robotic domains, hierarchical decomposition can simplify planning by using tools for low-level control given high-level action primitives (options). We introduce Constrained Options Belief Tree Search (COBeTS) to leverage this hierarchy and scale online search-based CPOMDP planning to large robotic problems. We show that if primitive option controllers are defined to satisfy assigned constraint budgets, then COBeTS will satisfy constraints anytime. Otherwise, COBeTS will guide the search towards a safe sequence of option primitives, and hierarchical monitoring can be used to achieve runtime safety. We demonstrate COBeTS in several safety-critical, constrained partially observable robotic domains, showing that it can plan successfully in continuous CPOMDPs while non-hierarchical baselines cannot.