mlsanityData to 7 Oct 2026

MAGICS: Adversarial RL with Minimax Actors Guided by Implicit Critic Stackelberg for Convergent Neural Synthesis of Robot Safety

arXiv 2409.13867 ↗First posted 20 Sep 2024cs.RO cs.AI cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

While robust optimal control theory provides a rigorous framework to compute robot control policies that are provably safe, it struggles to scale to high-dimensional problems, leading to increased use of deep learning for tractable synthesis of robot safety. Unfortunately, existing neural safety synthesis methods often lack convergence guarantees and solution interpretability. In this paper, we present Minimax Actors Guided by Implicit Critic Stackelberg (MAGICS), a novel adversarial reinforcement learning (RL) algorithm that guarantees local convergence to a minimax equilibrium solution. We then build on this approach to provide local convergence guarantees for a general deep RL-based robot safety synthesis algorithm. Through both simulation studies on OpenAI Gym environments and hardware experiments with a 36-dimensional quadruped robot, we show that MAGICS can yield robust control policies outperforming the state-of-the-art neural safety synthesis methods.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 20 Sep 2024.

Builds on

33 of 52 references.

References · 33

  1. 2022Generative Adversarial Networkscited by 5,603
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2016OpenAI Gymcited by 1,684
  6. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  7. 2017Robust Adversarial Reinforcement Learningcited by 439
  8. 2021RMA: Rapid Motor Adaptation for Legged Robotscited by 430
  9. 2015Gradient-based Hyperparameter Optimization through Reversible Learningcited by 357
  10. 2016Entropy-SGD: Biasing Gradient Descent Into Wide Valleyscited by 357
  11. 2017Learning with Opponent-Learning Awarenesscited by 218
  12. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 186
21 more
  1. 2017A General Safety Framework for Learning-Based Control in Uncertain Robotic Systemscited by 150
  2. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 91
  3. 2020Learning Control Barrier Functions from Expert Demonstrationscited by 88
  4. 2020DeepReach: A Deep Learning Approach to High-Dimensional Reachabilitycited by 81
  5. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  6. 2021Who Leads and Who Follows in Strategic Classification?cited by 53
  7. 2024Agile But Safe: Learning Collision-Free High-Speed Legged Locomotioncited by 53
  8. 2020Safety-Critical Model Predictive Control with Discrete-Time Control Barrier Functioncited by 42
  9. 2022Robust Reinforcement Learning as a Stackelberg Game via Adaptively-Regularized Adversarial Trainingcited by 26
  10. 2022ISAACS: Iterative Soft Adversarial Actor-Critic for Safetycited by 24
  11. 2024GCBF+: A Neural Graph Control Barrier Function Framework for Distributed Safe Multi-Agent Controlcited by 20
  12. 2021Stackelberg Actor-Critic: Game-Theoretic Reinforcement Learning Algorithmscited by 18
  13. 2021A predictive safety filter for learning-based racing controlcited by 17
  14. 2021FaSTrack: a Modular Framework for Real-Time Motion Planning and Guaranteed Safe Trackingcited by 15
  15. 2023Active Uncertainty Reduction for Safe and Efficient Interaction Planning: A Shielding-Aware Dual Control Approachcited by 13
  16. 2018Approximate Fisher Information Matrix to Characterise the Training of Deep Neural Networkscited by 12
  17. 2021SHARP: Shielding-Aware Robust Planning for Safe and Efficient Human-Robot Interactioncited by 12
  18. 2022Sim-to-Real Transfer for Quadrupedal Locomotion via Terrain Transformercited by 12
  19. 2023Deception Game: Closing the Safety-Learning Loop in Interactive Robot Autonomycited by 9
  20. 2022Onboard Safety Guarantees for Racing Drones: High-speed Geofencing with Control Barrier Functionscited by 3
  21. 2024ELA: Exploited Level Augmentation for Offline Learning in Zero-Sum Gamescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024Safe Multi-Agent Reinforcement Learning with Convergence to Generalized Nash Equilibriumcited by 5
  2. 2025Provably Optimal Reinforcement Learning under Safety Filteringcited by 2
  3. 2026Permissive Safety Through Trusted Inference: Verifiable Belief-Space Neural Safety Filters for Assured Interactive Roboticscited by 1
  4. 2025Riemannian Manifold Learning for Stackelberg Games with Neural Flow Representationscited by 0
  5. 2026Turning Safety into Competence: Minimally Exploitable Robot Policies via Safety-Filtered Reinforcement Learningcited by 0