# Recursively-Constrained Partially Observable Markov Decision Processes

arXiv: https://arxiv.org/abs/2310.09688 · HTML: https://mlsanity.com/paper/2310.09688/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 42 references that are in the map.
- First posted on arXiv 15 Oct 2023.

## Builds on (most cited first, 5 of 5)

- [Partially Observable Markov Decision Processes in Robotics: A Survey](https://mlsanity.com/paper/2209.10342/) (2022)
- [Trust-Aware Decision Making for Human-Robot Collaboration: Model Learning and Planning](https://mlsanity.com/paper/1801.04099/) (2018)
- [Optimal Control of Partially Observable Markov Decision Processes with Finite Linear Temporal Logic Constraints](https://mlsanity.com/paper/2203.09038/) (2022)
- [Online Planning for Constrained POMDPs with Continuous Spaces through Dual Ascent](https://mlsanity.com/paper/2212.12154/) (2022)
- [Recursive Constraints to Prevent Instability in Constrained Reinforcement Learning](https://mlsanity.com/paper/2201.07958/) (2022)

## Built on by (most cited first, 2 of 2)

- [Anytime Probabilistically Constrained Provably Convergent Online Belief Space Planning](https://mlsanity.com/paper/2411.06711/) (2024)
- [Distinguish or Homogenize: Last-Chance Policy Identification and Risk-Budgeted Recovery under Irreversible Resource Depletion](https://mlsanity.com/paper/2609.36741/) (2026)

## Abstract (arXiv)

Many sequential decision problems involve optimizing one objective function while imposing constraints on other objectives. Constrained Partially Observable Markov Decision Processes (C-POMDP) model this case with transition uncertainty and partial observability. In this work, we first show that C-POMDPs violate the optimal substructure property over successive decision steps and thus may exhibit behaviors that are undesirable for some (e.g., safety critical) applications. Additionally, online re-planning in C-POMDPs is often ineffective due to the inconsistency resulting from this violation. To address these drawbacks, we introduce the Recursively-Constrained POMDP (RC-POMDP), which imposes additional history-dependent cost constraints on the C-POMDP. We show that, unlike C-POMDPs, RC-POMDPs always have deterministic optimal policies and that optimal policies obey Bellman's principle of optimality. We also present a point-based dynamic programming algorithm for RC-POMDPs. Evaluations on benchmark problems demonstrate the efficacy of our algorithm and show that policies for RC-POMDPs produce more desirable behaviors than policies for C-POMDPs.