# Gameplay Filters: Robust Zero-Shot Safety through Adversarial Imagination

arXiv: https://arxiv.org/abs/2405.00846 · HTML: https://mlsanity.com/paper/2405.00846/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 52 references that are in the map.
- First posted on arXiv 1 May 2024.

## Builds on (most cited first, 30 of 40)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Control Barrier Functions: Theory and Applications](https://mlsanity.com/paper/1903.11199/) (2019)
- [Robots That Ask For Help: Uncertainty Alignment for Large Language Model Planners](https://mlsanity.com/paper/2307.01928/) (2023)
- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [Recovery RL: Safe Reinforcement Learning with Learned Recovery Zones](https://mlsanity.com/paper/2010.15920/) (2020)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Robot Parkour Learning](https://mlsanity.com/paper/2309.05665/) (2023)
- [A predictive safety filter for learning-based control of constrained nonlinear dynamical systems](https://mlsanity.com/paper/1812.05506/) (2018)
- [Conservative Safety Critics for Exploration](https://mlsanity.com/paper/2010.14497/) (2020)
- [Learning Control Barrier Functions from Expert Demonstrations](https://mlsanity.com/paper/2004.03315/) (2020)
- [DeepReach: A Deep Learning Approach to High-Dimensional Reachability](https://mlsanity.com/paper/2011.02082/) (2020)
- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [Agile But Safe: Learning Collision-Free High-Speed Legged Locomotion](https://mlsanity.com/paper/2401.17583/) (2024)
- [Robust Control Barrier-Value Functions for Safety-Critical Control](https://mlsanity.com/paper/2104.02808/) (2021)
- [Safety and Liveness Guarantees through Reach-Avoid Reinforcement Learning](https://mlsanity.com/paper/2112.12288/) (2021)
- [Reach-Avoid Problems with Time-Varying Dynamics, Targets and Constraints](https://mlsanity.com/paper/1410.6445/) (2014)
- [Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programming](https://mlsanity.com/paper/2004.07876/) (2020)
- [Linear model predictive safety certification for learning-based control](https://mlsanity.com/paper/1803.08552/) (2018)
- [Safe Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2203.02638/) (2022)
- [Robustness of Control Barrier Functions for Safety Critical Control](https://mlsanity.com/paper/1612.01554/) (2016)
- [ISAACS: Iterative Soft Adversarial Actor-Critic for Safety](https://mlsanity.com/paper/2212.03228/) (2022)
- [Run Time Assurance for Safety-Critical Systems: An Introduction to Safety Filtering Approaches for Complex Control Systems](https://mlsanity.com/paper/2110.03506/) (2021)
- [Sim-to-Lab-to-Real: Safe Reinforcement Learning with Shielding and Generalization Guarantees](https://mlsanity.com/paper/2201.08355/) (2022)
- [Learning Hybrid Control Barrier Functions from Data](https://mlsanity.com/paper/2011.04112/) (2020)
- [Safe Occlusion-aware Autonomous Driving via Game-Theoretic Active Perception](https://mlsanity.com/paper/2105.08169/) (2021)
- [Learning and Deploying Robust Locomotion Policies with Minimal Dynamics Randomization](https://mlsanity.com/paper/2209.12878/) (2022)
- [Permissive Barrier Certificates for Safe Stabilization Using Sum-of-squares](https://mlsanity.com/paper/1802.08917/) (2018)
- [Learning Robust Hybrid Control Barrier Functions for Uncertain Systems](https://mlsanity.com/paper/2101.06492/) (2021)
- [Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformer](https://mlsanity.com/paper/2212.07740/) (2022)

## Built on by (most cited first, 16 of 16)

- [Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failures](https://mlsanity.com/paper/2505.00779/) (2025)
- [Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning](https://mlsanity.com/paper/2604.13192/) (2026)
- [Provably Optimal Reinforcement Learning under Safety Filtering](https://mlsanity.com/paper/2510.18082/) (2025)
- [Neural Fidelity Calibration for Informative Sim-to-Real Adaptation](https://mlsanity.com/paper/2504.08604/) (2025)
- [Safety-Aware Imitation Learning via MPC-Guided Disturbance Injection](https://mlsanity.com/paper/2508.03129/) (2025)
- [Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Robotics](https://mlsanity.com/paper/2606.02562/) (2026)
- [Selective Agentic Recovery for UAV Autonomy with a Persistent Mission Runtime](https://mlsanity.com/paper/2606.14219/) (2026)
- [Efficient Coordination with the System-Level Shared State: An Embodied-AI Native Modular Framework](https://mlsanity.com/paper/2601.13945/) (2026)
- [Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learning](https://mlsanity.com/paper/2602.15817/) (2026)
- [Shield-Loco: Shielding Locomotion Policies with Predictive Safety Filtering](https://mlsanity.com/paper/2606.07193/) (2026)
- [Directional Constraints for Efficient Exploration in Safe Reinforcement Learning](https://mlsanity.com/paper/2607.12784/) (2026)
- [No Free Checker: A Survey of Verifiers for Robot Policies](https://mlsanity.com/paper/2609.09250/) (2026)
- [ResSafe: Learning Safety Filtering with Residual Reinforcement Learning for Humanoids](https://mlsanity.com/paper/2609.15988/) (2026)
- [Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning](https://mlsanity.com/paper/2609.27312/) (2026)
- [LEAP-CBF: A Safety Filter for Uncertain Systems with Least-Effort Adversarial Potentials](https://mlsanity.com/paper/2609.28364/) (2026)
- [ORACLE: Optimizer-Relative Alignment for Constrained LEarning](https://mlsanity.com/paper/2610.09040/) (2026)

## Abstract (arXiv)

Despite the impressive recent advances in learning-based robot control, ensuring robustness to out-of-distribution conditions remains an open challenge. Safety filters can, in principle, keep arbitrary control policies from incurring catastrophic failures by overriding unsafe actions, but existing solutions for complex (e.g., legged) robot dynamics do not span the full motion envelope and instead rely on local, reduced-order models. These filters tend to overly restrict agility and can still fail when perturbed away from nominal conditions. This paper presents the gameplay filter, a new class of predictive safety filter that continually plays out hypothetical matches between its simulation-trained safety strategy and a virtual adversary co-trained to invoke worst-case events and sim-to-real error, and precludes actions that would cause failures down the line. We demonstrate the scalability and robustness of the approach with a first-of-its-kind full-order safety filter for (36-D) quadrupedal dynamics. Physical experiments on two different quadruped platforms demonstrate the superior zero-shot effectiveness of the gameplay filter under large perturbations such as tugging and unmodeled terrain. Experiment videos and open-source software are available online: https://saferobotics.org/research/gameplay-filter