mlsanityData to 7 Oct 2026

Discrete GCBF Proximal Policy Optimization for Multi-agent Safe Optimal Control

arXiv 2502.03640 ↗First posted 5 Feb 2025cs.RO cs.LG cs.MAcited by 7

Not on the map. Why?

Abstract (arXiv)

Control policies that can achieve high task performance and satisfy safety constraints are desirable for any system, including multi-agent systems (MAS). One promising technique for ensuring the safety of MAS is distributed control barrier functions (CBF). However, it is difficult to design distributed CBF-based policies for MAS that can tackle unknown discrete-time dynamics, partial observability, changing neighborhoods, and input constraints, especially when a distributed high-performance nominal policy that can achieve the task is unavailable. To tackle these challenges, we propose DGPPO, a new framework that simultaneously learns both a discrete graph CBF which handles neighborhood changes and input constraints, and a distributed high-performance safe policy for MAS with unknown discrete-time dynamics. We empirically validate our claims on a suite of multi-agent tasks spanning three different simulation engines. The results suggest that, compared with existing methods, our DGPPO framework obtains policies that achieve high task performance (matching baselines that ignore the safety constraints), and high safety rates (matching the most conservative baselines), with a constant set of hyperparameters across all environments.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 52 of its 73 references that are in the map. First posted on arXiv 5 Feb 2025.

Builds on

52 of 73 references.

References · 52

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2017Graph Attention Networkscited by 8,854
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  4. 2020Gradient Surgery for Multi-Task Learningcited by 1,210
  5. 2017Constrained Policy Optimizationcited by 928
  6. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 903
  7. 2020Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classificationcited by 500
  8. 2021Conflict-Averse Gradient Descent for Multi-task Learningcited by 441
  9. 2018Reward Constrained Policy Optimizationcited by 397
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  11. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  12. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
38 more
  1. 2017Socially Aware Motion Planning with Deep Reinforcement Learningcited by 128
  2. 2020CRPO: A New Approach for Safe Reinforcement Learning with Convergence Guaranteecited by 128
  3. 2016Decentralized Non-communicating Multiagent Collision Avoidance with Deep Reinforcement Learningcited by 104
  4. 2018Motion Planning Among Dynamic, Decision-Making Agents with Deep Reinforcement Learningcited by 101
  5. 2021Safe Nonlinear Control Using Robust Neural Lyapunov-Barrier Functionscited by 91
  6. 2017Towards Optimally Decentralized Multi-Robot Collision Avoidance via Deep Reinforcement Learningcited by 81
  7. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 77
  8. 2019Safe Reinforcement Learning Using Robust MPCcited by 72
  9. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 71
  10. 2022VMAS: A Vectorized Multi-Agent Simulator for Collective Robot Learningcited by 63
  11. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 50
  12. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 44
  13. 2023How to Train Your Neural Control Barrier Function: Learning Safety Filters for Complex Input-Constrained Systemscited by 40
  14. 2020Synthesis of Control Barrier Functions Using a Supervised Machine Learning Approachcited by 35
  15. 2022Scalable Multi-Agent Reinforcement Learning through Intelligent Information Aggregationcited by 34
  16. 2023Iterative Reachability Estimation for Safe Reinforcement Learningcited by 33
  17. 2021Multi-Agent Constrained Policy Optimisationcited by 32
  18. 2023BenchMARL: Benchmarking Multi-Agent Reinforcement Learningcited by 31
  19. 2019Control Barrier Functions for Systems with High Relative Degreecited by 30
  20. 2016Robustness of Control Barrier Functions for Safety Critical Controlcited by 29
  21. 2020Learning Hybrid Control Barrier Functions from Datacited by 24
  22. 2023Solving Stabilize-Avoid Optimal Control via Epigraph Form and Deep Reinforcement Learningcited by 24
  23. 2020Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEscited by 22
  24. 2020Multi-agent Motion Planning for Dense and Dynamic Environments via Deep Reinforcement Learningcited by 21
  25. 2024GCBF+: A Neural Graph Control Barrier Function Framework for Distributed Safe Multi-Agent Controlcited by 20
  26. 2020Learning Barrier Functions for Constrained Motion Planning with Dynamical Systemscited by 17
  27. 2021A predictive safety filter for learning-based racing controlcited by 17
  28. 2023AutoCost: Evolving Intrinsic Cost for Zero-violation Reinforcement Learningcited by 16
  29. 2021PowerGridworld: A Framework for Multi-Agent Reinforcement Learning in Power Systemscited by 15
  30. 2023Provably Efficient Generalized Lagrangian Policy Optimization for Safe Multi-Agent Reinforcement Learningcited by 13
  31. 2023Learning Safe Control for Multi-Robot Systems: Methods, Verification, and Open Challengescited by 12
  32. 2019Safe Policy Synthesis in Multi-Agent POMDPs via Discrete-Time Barrier Functionscited by 11
  33. 2021Backup Control Barrier Functions: Formulation and Comparative Studycited by 10
  34. 2023Neural Graph Control Barrier Functions Guided Distributed Collision-avoidance Multi-agent Controlcited by 8
  35. 2021Safe Value Functionscited by 7
  36. 2020Control Barrier Function based Quadratic Programs Introduce Undesirable Asymptotically Stable Equilibriacited by 3
  37. 2022Decentralized Safe Multi-agent Stochastic Optimal Control using Deep FBSDEs and ADMMcited by 3
  38. 2024Optimal control barrier functions for RL based safe powertrain controlcited by 3

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 4.

7 papers.

  1. 2025Solving Multi-Agent Safe Optimal Control with Distributed Epigraph Form MARLcited by 4
  2. 2025Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systemscited by 4
  3. 2026How Does the Lagrangian Guide Safe Reinforcement Learning through Diffusion Models?cited by 4
  4. 2025Distributed Risk-Sensitive Safety Filters for Uncertain Discrete-Time Systemscited by 0
  5. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Controlcited by 0
  6. 2026Generalizable Multi-Agent Planning from Signal Temporal Logic Specifications via Diffusioncited by 0
  7. 2026Safe Score Matching: Diffusion Policies with Hamilton-Jacobi Reachability for Online Safe Reinforcement Learningcited by 0