# Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent

arXiv: https://arxiv.org/abs/2212.12154 · HTML: https://mlsanity.com/paper/2212.12154/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 24 references that are in the map.
- First posted on arXiv 23 Dec 2022.

## Builds on (most cited first, 5 of 5)

- [A Survey of Multi-Objective Sequential Decision-Making](https://mlsanity.com/paper/1402.0590/) (2014)
- [Online algorithms for POMDPs with continuous state, action, and observation spaces](https://mlsanity.com/paper/1709.06196/) (2017)
- [Online Planning Algorithms for POMDPs](https://mlsanity.com/paper/1401.3436/) (2014)
- [Bayesian Optimized Monte Carlo Planning](https://mlsanity.com/paper/2010.03597/) (2020)
- [A POMDP Model for Safe Geological Carbon Sequestration](https://mlsanity.com/paper/2212.00669/) (2022)

## Built on by (most cited first, 5 of 5)

- [ConstrainedZero: Chance-Constrained POMDP Planning using Learned Probabilistic Failure Surrogates and Adaptive Safety Constraints](https://mlsanity.com/paper/2405.00644/) (2024)
- [Recursively-Constrained Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2310.09688/) (2023)
- [Constrained Hierarchical Monte Carlo Belief-State Planning](https://mlsanity.com/paper/2310.20054/) (2023)
- [Addressing Myopic Constrained POMDP Planning with Recursive Dual Ascent](https://mlsanity.com/paper/2403.17358/) (2024)
- [Anytime Probabilistically Constrained Provably Convergent Online Belief Space Planning](https://mlsanity.com/paper/2411.06711/) (2024)

## Abstract (arXiv)

Rather than augmenting rewards with penalties for undesired behavior, Constrained Partially Observable Markov Decision Processes (CPOMDPs) plan safely by imposing inviolable hard constraint value budgets. Previous work performing online planning for CPOMDPs has only been applied to discrete action and observation spaces. In this work, we propose algorithms for online CPOMDP planning for continuous state, action, and observation spaces by combining dual ascent with progressive widening. We empirically compare the effectiveness of our proposed algorithms on continuous CPOMDPs that model both toy and real-world safety-critical problems. Additionally, we compare against the use of online solvers for continuous unconstrained POMDPs that scalarize cost constraints into rewards, and investigate the effect of optimistic cost propagation.