# Adaptive GR(1) Specification Repair for Liveness-Preserving Shielding in Reinforcement Learning

arXiv: https://arxiv.org/abs/2511.02605 · HTML: https://mlsanity.com/paper/2511.02605/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 53 references that are in the map.
- First posted on arXiv 4 Nov 2025.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 24 of 24)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Revisiting the Arcade Learning Environment: Evaluation Protocols and Open Problems for General Agents](https://mlsanity.com/paper/1709.06009/) (2017)
- [Reinforcement learning](https://mlsanity.com/paper/2405.10369/) (2024)
- [Conservative Safety Critics for Exploration](https://mlsanity.com/paper/2010.14497/) (2020)
- [OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real World](https://mlsanity.com/paper/1709.07643/) (2017)
- [Safe Reinforcement Learning by Imagining the Near Future](https://mlsanity.com/paper/2202.07789/) (2022)
- [Shield Synthesis: Runtime Enforcement for Reactive Systems](https://mlsanity.com/paper/1501.02573/) (2015)
- [Reset-free Trial-and-Error Learning for Robot Damage Recovery](https://mlsanity.com/paper/1610.04213/) (2016)
- [Online Shielding for Reinforcement Learning](https://mlsanity.com/paper/2212.01861/) (2022)
- [Adaptive Shielding under Uncertainty](https://mlsanity.com/paper/2010.03842/) (2020)
- [Dynamic Shielding for Reinforcement Learning in Black-Box Environments](https://mlsanity.com/paper/2207.13446/) (2022)
- [Model-based Dynamic Shielding for Safe and Efficient Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2304.06281/) (2023)
- [Do Androids Dream of Electric Fences? Safety-Aware Reinforcement Learning with Latent Shielding](https://mlsanity.com/paper/2112.11490/) (2021)
- [UAVs Beneath the Surface: Cooperative Autonomy for Subterranean Search and Rescue in DARPA SubT](https://mlsanity.com/paper/2206.08185/) (2022)
- [Explainable Artificial Intelligence (XAI): An Engineering Perspective](https://mlsanity.com/paper/2101.03613/) (2021)
- [Approximate Model-Based Shielding for Safe Reinforcement Learning](https://mlsanity.com/paper/2308.00707/) (2023)
- [Adaptive Shielding via Parametric Safety Proofs](https://mlsanity.com/paper/2502.18879/) (2025)
- [It's Time to Play Safe: Shield Synthesis for Timed Systems](https://mlsanity.com/paper/2006.16688/) (2020)
- [Fast Damage Recovery in Robotics with the T-Resilience Algorithm](https://mlsanity.com/paper/1302.0386/) (2013)
- [Spectra: A Specification Language for Reactive Systems](https://mlsanity.com/paper/1904.06668/) (2019)

## Abstract (arXiv)

Shielding is widely used to enforce safety in reinforcement learning (RL), ensuring that an agent's actions remain compliant with formal specifications. Classical shielding approaches, however, are often static, in the sense that they assume fixed logical specifications and hand-crafted abstractions. While these static shields provide safety under nominal assumptions, they fail to adapt when environment assumptions are violated. In this paper, we develop an adaptive shielding framework based on based on Generalized Reactivity of rank 1 (GR(1)) specifications, a tractable and expressive fragment of Linear Temporal Logic (LTL) that captures both safety and liveness properties. Our method detects environment assumption violations at runtime and employs Inductive Logic Programming (ILP) to automatically repair GR(1) specifications online, in a systematic and interpretable way. This ensures that the shield evolves gracefully, ensuring liveness is achievable and minimally weakening goals only when necessary. We consider two case studies: Minepump and Atari Seaquest; showing that (i) static symbolic controllers are often severely suboptimal when optimizing for auxiliary rewards, and (ii) RL agents equipped with our adaptive shield maintain near-optimal reward and perfect logical compliance compared with static shields.