mlsanityData to 14 Oct 2026

Hierarchical Multi-Agent Reinforcement Learning with Control Barrier Functions for Safety-Critical Autonomous Systems

arXiv 2507.14850 ↗First posted 20 Jul 2025cs.LG cs.AI cs.ROcited by 5

Not on the map. Why?

Abstract (arXiv)

We address the problem of safe policy learning in multi-agent safety-critical autonomous systems. In such systems, it is necessary for each agent to meet the safety requirements at all times while also cooperating with other agents to accomplish the task. Toward this end, we propose a safe Hierarchical Multi-Agent Reinforcement Learning (HMARL) approach based on Control Barrier Functions (CBFs). Our proposed hierarchical approach decomposes the overall reinforcement learning problem into two levels learning joint cooperative behavior at the higher level and learning safe individual behavior at the lower or agent level conditioned on the high-level policy. Specifically, we propose a skill-based HMARL-CBF algorithm in which the higher level problem involves learning a joint policy over the skills for all the agents and the lower-level problem involves learning policies to execute the skills safely with CBFs. We validate our approach on challenging environment scenarios whereby a large number of agents have to safely navigate through conflicting road networks. Compared with existing state of the art methods, our approach significantly improves the safety achieving near perfect (within 5%) success/safety rate while also improving performance across all the environments.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 58 of its 90 references that are in the map. First posted on arXiv 20 Jul 2025.

Builds on

58 of 90 references.

References · 58

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,558
  4. 2021The Surprising Effectiveness of PPO in Cooperative, Multi-Agent Gamescited by 942
  5. 2017Counterfactual Multi-Agent Policy Gradientscited by 751
  6. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 708
  7. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  8. 2017Value-Decomposition Networks For Cooperative Multi-Agent Learningcited by 503
  9. 2020Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 485
  10. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 455
  11. 2016The Option-Critic Architecturecited by 448
  12. 2019QTRAN: Learning to Factorize with Transformation for Cooperative Multi-Agent Reinforcement Learningcited by 379
38 more
  1. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  2. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 337
  3. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  4. 2020Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?cited by 294
  5. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  6. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 286
  7. 2016Variational Intrinsic Controlcited by 267
  8. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 261
  9. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  10. 2018Mean Field Multi-Agent Reinforcement Learningcited by 207
  11. 2019Lyapunov-based Safe Policy Optimization for Continuous Controlcited by 145
  12. 2019IPO: Interior-point Policy Optimization under Constraintscited by 132
  13. 2016A Deep Hierarchical Approach to Lifelong Learning in Minecraftcited by 90
  14. 2021Learning Safe Multi-Agent Control with Decentralized Neural Barrier Certificatescited by 74
  15. 2017Flow: A Modular Learning Framework for Mixed Autonomy Trafficcited by 73
  16. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  17. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 59
  18. 2022Reachability Constrained Reinforcement Learningcited by 59
  19. 2017When Waiting is not an Option : Learning Options with a Deliberation Costcited by 58
  20. 2023SafeDreamer: Safe Reinforcement Learning with World Modelscited by 54
  21. 2021Robust Control Barrier-Value Functions for Safety-Critical Controlcited by 51
  22. 2018Variance reduction properties of the reparameterization trickcited by 41
  23. 2019Hierarchical Reinforcement Learning with Advantage-Based Auxiliary Rewardscited by 40
  24. 2022Differentiable Simulation of Soft Multi-body Systemscited by 39
  25. 2021Hierarchical Skills for Efficient Explorationcited by 38
  26. 2022Skill-based Meta-Reinforcement Learningcited by 37
  27. 2019Feudal Multi-Agent Hierarchies for Cooperative Reinforcement Learningcited by 33
  28. 2021Learning to Simulate Self-Driven Particles System with Coordinated Policy Optimizationcited by 33
  29. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 30
  30. 2024Reasoning Multi-Agent Behavioral Topology for Interactive Autonomous Drivingcited by 27
  31. 2020Safe Optimal Control Using Stochastic Barrier Functions and Deep Forward-Backward SDEscited by 22
  32. 2021HAVEN: Hierarchical Cooperative Multi-Agent Reinforcement Learning with Dual Coordination Mechanismcited by 19
  33. 2020RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problemcited by 16
  34. 2024Solving Minimum-Cost Reach Avoid using Reinforcement Learningcited by 14
  35. 2019Multi-Robot Collision Avoidance under Uncertainty with Probabilistic Safety Barrier Certificatescited by 13
  36. 2022Rethinking Individual Global Max in Cooperative Multi-Agent Reinforcement Learningcited by 13
  37. 2016Adaptive Skills, Adaptive Partitions (ASAP)cited by 11
  38. 2022Safe and Efficient Reinforcement Learning Using Disturbance-Observer-Based Control Barrier Functionscited by 11

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 4.

5 papers.

  1. 2025Goal Reaching with Eikonal-Constrained Hierarchical Quasimetric Reinforcement Learningcited by 6
  2. 2026Decentralized Autonomous Traffic Management through Corridor Networkscited by 0
  3. 2026Safe and Generalizable Hierarchical Multi-Agent RL via Constraint Manifold Controlcited by 0
  4. 2026Lightweight Safe Reinforcement Learning for End-to-End UAV Navigationcited by 0
  5. 2026Position: Beyond Training Time, Test-Time Coordination is Essential for Cooperative MARLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Certificated Actor-Critic: Hierarchical Reinforcement Learning with Control Barrier Functions for Safe Navigation
  2. 2026Individual Control Barrier Functions-Guided Diffusion Model for Safe Offline Multi-Agent Reinforcement Learning
  3. 2025Resolving Conflicting Constraints in Multi-Agent Reinforcement Learning with Layered Safety
  4. 2025Safe Multi-Agent Navigation guided by Goal-Conditioned Safe Reinforcement Learning
  5. 2025Tackling Uncertainties in Multi-Agent Reinforcement Learning through Integration of Agent Termination Dynamics
  6. 2024Enforcing Cooperative Safety for Reinforcement Learning-based Mixed-Autonomy Platoon Control
  7. 2024Safety-Aware Multi-Agent Learning for Dynamic Network Bridging
  8. 2021A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning