# Shielding in Resource-Constrained Goal POMDPs

arXiv: https://arxiv.org/abs/2211.15349 · HTML: https://mlsanity.com/paper/2211.15349/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 28 Nov 2022.

## Builds on (most cited first, 12 of 12)

- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Safety-Constrained Reinforcement Learning for MDPs](https://mlsanity.com/paper/1510.05880/) (2015)
- [Permissive Controller Synthesis for Probabilistic Systems](https://mlsanity.com/paper/1504.04662/) (2015)
- [Enforcing Almost-Sure Reachability in POMDPs](https://mlsanity.com/paper/2007.00085/) (2020)
- [Optimal Cost Almost-sure Reachability in POMDPs](https://mlsanity.com/paper/1411.3880/) (2014)
- [Qualitative Controller Synthesis for Consumption Markov Decision Processes](https://mlsanity.com/paper/2005.07227/) (2020)
- [Reinforcement Learning for POMDP: Partitioned Rollout and Policy Iteration with Application to Autonomous Sequential Repair Problems](https://mlsanity.com/paper/2002.04175/) (2020)
- [Optimizing Expectation with Guarantees in POMDPs (Technical Report)](https://mlsanity.com/paper/1611.08696/) (2016)
- [Minimizing Running Costs in Consumption Systems](https://mlsanity.com/paper/1402.4995/) (2014)
- [Stochastic Shortest Path with Energy Constraints in POMDPs](https://mlsanity.com/paper/1602.07565/) (2016)
- [Optimizing the Expected Mean Payoff in Energy Markov Decision Processes](https://mlsanity.com/paper/1607.00678/) (2016)
- [MDPs with Energy-Parity Objectives](https://mlsanity.com/paper/1701.02546/) (2017)

## Built on by (most cited first, 4 of 4)

- [Leveraging Approximate Model-based Shielding for Probabilistic Safety Guarantees in Continuous Environments](https://mlsanity.com/paper/2402.00816/) (2024)
- [Approximate Shielding of Atari Agents for Safe Exploration](https://mlsanity.com/paper/2304.11104/) (2023)
- [Anytime Probabilistically Constrained Provably Convergent Online Belief Space Planning](https://mlsanity.com/paper/2411.06711/) (2024)
- [Distinguish or Homogenize: Last-Chance Policy Identification and Risk-Budgeted Recovery under Irreversible Resource Depletion](https://mlsanity.com/paper/2609.36741/) (2026)

## Abstract (arXiv)

We consider partially observable Markov decision processes (POMDPs) modeling an agent that needs a supply of a certain resource (e.g., electricity stored in batteries) to operate correctly. The resource is consumed by agent's actions and can be replenished only in certain states. The agent aims to minimize the expected cost of reaching some goal while preventing resource exhaustion, a problem we call resource-constrained goal optimization (RSGO). We take a two-step approach to the RSGO problem. First, using formal methods techniques, we design an algorithm computing a shield for a given scenario: a procedure that observes the agent and prevents it from using actions that might eventually lead to resource exhaustion. Second, we augment the POMCP heuristic search algorithm for POMDP planning with our shields to obtain an algorithm solving the RSGO problem. We implement our algorithm and present experiments showing its applicability to benchmarks from the literature.