# Synthesis and Deployment of Maximal Robust Control Barrier Functions through Adversarial Reinforcement Learning

arXiv: https://arxiv.org/abs/2604.13192 · HTML: https://mlsanity.com/paper/2604.13192/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 30 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Placed in

- Deep reinforcement learning methods / [Safe and constrained RL](https://mlsanity.com/deeprl/safe-and-constrained-rl/)

## Builds on (most cited first, 22 of 22)

- [Hamilton-Jacobi Reachability: A Brief Overview and Recent Advances](https://mlsanity.com/paper/1709.07523/) (2017)
- [The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systems](https://mlsanity.com/paper/2309.05837/) (2023)
- [Robust Control Barrier-Value Functions for Safety-Critical Control](https://mlsanity.com/paper/2104.02808/) (2021)
- [ISAACS: Iterative Soft Adversarial Actor-Critic for Safety](https://mlsanity.com/paper/2212.03228/) (2022)
- [Verification of Neural Reachable Tubes via Scenario Optimization and Conformal Prediction](https://mlsanity.com/paper/2312.08604/) (2023)
- [Learning Robust Output Control Barrier Functions from Safe Expert Demonstrations](https://mlsanity.com/paper/2111.09971/) (2021)
- [Gameplay Filters: Robust Zero-Shot Safety through Adversarial Imagination](https://mlsanity.com/paper/2405.00846/) (2024)
- [Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approach](https://mlsanity.com/paper/2302.00171/) (2023)
- [Towards Robust Data-Driven Control Synthesis for Nonlinear Systems with Actuation Uncertainty](https://mlsanity.com/paper/2011.10730/) (2020)
- [OptimizedDP: An Efficient, User-friendly Library For Optimal Control and Dynamic Programming](https://mlsanity.com/paper/2204.05520/) (2022)
- [Robust Control Barrier Functions with Sector-Bounded Uncertainties](https://mlsanity.com/paper/2109.02537/) (2021)
- [Safety with Agency: Human-Centered Safety Filter with Application to AI-Assisted Motorsports](https://mlsanity.com/paper/2504.11717/) (2025)
- [MAGICS: Adversarial RL with Minimax Actors Guided by Implicit Critic Stackelberg for Convergent Neural Synthesis of Robot Safety](https://mlsanity.com/paper/2409.13867/) (2024)
- [Robust Control Barrier Functions for Nonlinear Control Systems with Uncertainty: A Duality-based Approach](https://mlsanity.com/paper/2208.05955/) (2022)
- [Multi-Step Model Predictive Safety Filters: Reducing Chattering by Increasing the Prediction Horizon](https://mlsanity.com/paper/2309.11453/) (2023)
- [Data-Driven Robust Barrier Functions for Safe, Long-Term Operation](https://mlsanity.com/paper/2104.07592/) (2021)
- [Disturbance Observers for Robust Safety-critical Control with Control Barrier Functions](https://mlsanity.com/paper/2209.08123/) (2022)
- [Provably Optimal Reinforcement Learning under Safety Filtering](https://mlsanity.com/paper/2510.18082/) (2025)
- [Parameterized Barrier Functions to Guarantee Safety under Uncertainty](https://mlsanity.com/paper/2303.11385/) (2023)
- [Safety-Critical Control under Multiple State and Input Constraints and Application to Fixed-Wing UAV](https://mlsanity.com/paper/2308.04087/) (2023)
- [Back to Base: Towards Hands-Off Learning via Safe Resets with Reach-Avoid Safety Filters](https://mlsanity.com/paper/2501.02620/) (2025)
- [Viscosity CBFs: Bridging the Control Barrier Function and Hamilton-Jacobi Reachability Frameworks in Safe Control Theory](https://mlsanity.com/paper/2510.09929/) (2025)

## Built on by (most cited first, 5 of 5)

- [Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learning](https://mlsanity.com/paper/2609.27312/) (2026)
- [LEAP-CBF: A Safety Filter for Uncertain Systems with Least-Effort Adversarial Potentials](https://mlsanity.com/paper/2609.28364/) (2026)
- [Modeling Latent Disturbances for Robust Decision-Making in World Models](https://mlsanity.com/paper/2610.07599/) (2026)
- [A Unified Bellman Operator for Safety-Critical Reinforcement Learning](https://mlsanity.com/paper/2610.12420/) (2026)
- [FAITH: Feasibility-Aware Safety-Filtered RL for High-Dimensional Systems](https://mlsanity.com/paper/2610.12432/) (2026)

## Abstract (arXiv)

Robust control barrier functions (CBFs) provide a principled mechanism for smooth safety enforcement under worst-case disturbances. However, existing approaches typically rely on explicit, closed-form structure in the dynamics (e.g., control-affine) and uncertainty models. This has led to limited scalability and generality, with most robust CBFs certifying only conservative subsets of the maximal robust safe set. In this paper, we introduce a new robust CBF framework for general nonlinear systems under bounded uncertainty. We first show that the safety value function solving the dynamic programming Isaacs equation is a valid robust discrete-time CBF that enforces safety on the maximal robust safe set. We then adopt the key reinforcement learning (RL) notion of quality function (or Q-function), which removes the need for explicit dynamics by lifting the barrier certificate into state-action space and yields a novel robust Q-CBF constraint for safety filtering. Combined with adversarial RL, this enables the synthesis and deployment of robust Q-CBFs on general nonlinear systems with black-box dynamics and unknown uncertainty structure. We validate the framework on a canonical inverted pendulum benchmark and a 36-D quadruped simulator, achieving substantially less conservative safe sets than barrier-based baselines on the pendulum and reliable safety enforcement even under adversarial uncertainty realizations on the quadruped.