# Follow the STARs: Dynamic ω-Regular Shielding of Learned Policies

arXiv: https://arxiv.org/abs/2505.14689 · HTML: https://mlsanity.com/paper/2505.14689/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 60 references that are in the map.
- First posted on arXiv 11 Apr 2025.

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [On the Utility of Learning about Humans for Human-AI Coordination](https://mlsanity.com/paper/1910.05789/) (2019)
- [Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systems](https://mlsanity.com/paper/2405.06624/) (2024)
- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [Safe Multi-Agent Reinforcement Learning via Shielding](https://mlsanity.com/paper/2101.11196/) (2021)
- [Safe Reinforcement Learning via Probabilistic Logic Shields](https://mlsanity.com/paper/2303.03226/) (2023)
- [On-Policy Deep Reinforcement Learning for the Average-Reward Criterion](https://mlsanity.com/paper/2106.07329/) (2021)
- [Online Shielding for Reinforcement Learning](https://mlsanity.com/paper/2212.01861/) (2022)
- [Correct-by-Construction Runtime Enforcement in AI -- A Survey](https://mlsanity.com/paper/2208.14426/) (2022)
- [Learning-Based Mean-Payoff Optimization in an Unknown MDP under Omega-Regular Constraints](https://mlsanity.com/paper/1804.08924/) (2018)
- [Multi-robot Mission Planning in Dynamic Semantic Environments](https://mlsanity.com/paper/2209.06323/) (2022)
- [Event-Based Signal Temporal Logic Synthesis for Single and Multi-Robot Tasks](https://mlsanity.com/paper/2011.00370/) (2020)
- [Runtime-Safety-Guided Policy Repair](https://mlsanity.com/paper/2008.07667/) (2020)
- [Physically-Feasible Repair of Reactive, Linear Temporal Logic-based, High-Level Tasks](https://mlsanity.com/paper/2207.02834/) (2022)
- [Context-triggered Abstraction-based Control Design](https://mlsanity.com/paper/2305.03399/) (2023)
- [Technical Report: Reactive Planning for Mobile Manipulation Tasks in Unexplored Semantic Environments](https://mlsanity.com/paper/2011.00642/) (2020)
- [Synthesizing Permissive Winning Strategy Templates for Parity Games](https://mlsanity.com/paper/2305.14026/) (2023)
- [Localized Attractor Computations for Infinite-State Games (Full Version)](https://mlsanity.com/paper/2405.09281/) (2024)
- [Safe Learning for Near Optimal Scheduling](https://mlsanity.com/paper/2005.09253/) (2020)
- [On Minimizing Total Discounted Cost in MDPs Subject to Reachability Constraints](https://mlsanity.com/paper/2103.09295/) (2021)
- [Omega-Regular Reward Machines](https://mlsanity.com/paper/2308.07469/) (2023)
- [Shielded Deep Reinforcement Learning for Complex Spacecraft Tasking](https://mlsanity.com/paper/2403.05693/) (2024)
- [Minimizing Expected Cost Under Hard Boolean Constraints, with Applications to Quantitative Synthesis](https://mlsanity.com/paper/1604.07064/) (2016)
- [Reactive Control Improvisation](https://mlsanity.com/paper/1804.05037/) (2018)
- [Reachability Games with Relaxed Energy Constraints](https://mlsanity.com/paper/1909.07653/) (2019)
- [Computing Adequately Permissive Assumptions for Synthesis](https://mlsanity.com/paper/2301.07563/) (2023)
- [Contract-Based Distributed Synthesis in Two-Objective Parity Games](https://mlsanity.com/paper/2307.06212/) (2023)

## Built on by (most cited first, 1 of 1)

- [MENTOR: Procedural Shielding of Tool-Using LLM Agents](https://mlsanity.com/paper/2610.17039/) (2026)

## Abstract (arXiv)

This paper presents a novel dynamic post-shielding framework that enforces the full class of ω-regular correctness properties over pre-computed probabilistic policies. This constitutes a paradigm shift from the predominant setting of safety-shielding -- i.e., ensuring that nothing bad ever happens -- to a shielding process that additionally enforces liveness -- i.e., ensures that something good eventually happens. At the core, our method uses Strategy-Template-based Adaptive Runtime Shields (STARs), which leverage permissive strategy templates to enable post-shielding with minimal interference. As its main feature, STARs introduce a mechanism to dynamically control interference, allowing a tunable enforcement parameter to balance formal obligations and task-specific behavior at runtime. This allows to trigger more aggressive enforcement when needed, while allowing for optimized policy choices otherwise. In addition, STARs support runtime adaptation to changing specifications or actuator failures, making them especially suited for cyber-physical applications. We evaluate STARs on a mobile robot benchmark to demonstrate their controllable interference when enforcing (incrementally updated) ω-regular correctness properties over learned probabilistic policies.