mlsanityData to 7 Oct 2026

Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios

arXiv 2210.02300 ↗First posted 5 Oct 2022cs.RO cs.AI cs.MAcited by 12

On the map: End-to-end autonomous driving → Cooperative and V2X driving

Abstract (arXiv)

Communication technologies enable coordination among connected and autonomous vehicles (CAVs). However, it remains unclear how to utilize shared information to improve the safety and efficiency of the CAV system in dynamic and complicated driving scenarios. In this work, we propose a framework of constrained multi-agent reinforcement learning (MARL) with a parallel Safety Shield for CAVs in challenging driving scenarios that includes unconnected hazard vehicles. The coordination mechanisms of the proposed MARL include information sharing and cooperative policy learning, with Graph Convolutional Network (GCN)-Transformer as a spatial-temporal encoder that enhances the agent's environment awareness. The Safety Shield module with Control Barrier Functions (CBF)-based safety checking protects the agents from taking unsafe actions. We design a constrained multi-agent advantage actor-critic (CMAA2C) algorithm to train safe and cooperative policies for CAVs. With the experiment deployed in the CARLA simulator, we verify the performance of the safety checking, spatial-temporal encoder, and coordination mechanisms designed in our method by comparative experiments in several challenging scenarios with unconnected hazard vehicles. Results show that our proposed methodology significantly increases system safety and efficiency in challenging scenarios.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving. Builds on 24 of its 37 references that are in the map. First posted on arXiv 5 Oct 2022.

Builds on

24 of 37 references.

References · 24

  1. 2021Convex optimizationcited by 2,479
  2. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2017Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environmentscited by 1,519
  5. 2016End to End Learning for Self-Driving Carscited by 1,000
  6. 2017Counterfactual Multi-Agent Policy Gradientscited by 733
  7. 2018QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learningcited by 697
  8. 2019Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithmscited by 555
  9. 2019Control Barrier Functions: Theory and Applicationscited by 332
  10. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  11. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 261
  12. 2018Actor-Attention-Critic for Multi-Agent Reinforcement Learningcited by 259
12 more
  1. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 256
  2. 2015DeepDriving: Learning Affordance for Direct Perception in Autonomous Drivingcited by 229
  3. 2017Virtual to Real Reinforcement Learning for Autonomous Drivingcited by 79
  4. 2019Robust Model Predictive Shielding for Safe Reinforcement Learning with Stochastic Dynamicscited by 45
  5. 2020Safe Reinforcement Learning for Autonomous Vehicles through Parallel Constrained Policy Optimizationcited by 34
  6. 2021Spatio-Temporal Scene-Graph Embedding for Autonomous Vehicle Collision Predictioncited by 29
  7. 2022CRAT-Pred: Vehicle Trajectory Prediction with Crystal Graph Convolutional Neural Networks and Multi-Head Self-Attentioncited by 21
  8. 2019MAMPS: Safe Multi-Agent Reinforcement Learning via Model Predictive Shieldingcited by 17
  9. 2020A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehiclescited by 12
  10. 2022Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehiclescited by 11
  11. 2021Rule-Based Safety-Critical Control Design using Control Barrier Functions with Application to Autonomous Lane Changecited by 9
  12. 2021Spatial-Channel Transformer Network for Trajectory Prediction on the Traffic Scenescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 6; 2025: 4; 2026: 1.

12 papers.

  1. 2024OASIS: Conditional Distribution Shaping for Offline Safe Reinforcement Learningcited by 10
  2. 2024Safe Multi-Agent Reinforcement Learning with Bilevel Optimization in Autonomous Drivingcited by 9
  3. 2024Efficient and Generalized end-to-end Autonomous Driving System with Latent Deep Reinforcement Learning and Demonstrationscited by 7
  4. 2023Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specificationscited by 6
  5. 2025High-order Interactions Modeling for Interpretable Multi-Agent Q-Learningcited by 2
  6. 2024Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Drivingcited by 1
  7. 2025Boundary-to-Region Supervision for Offline Safe Reinforcement Learningcited by 1
  8. 2024CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Communitycited by 0
  9. 2024AIR: Unifying Individual and Collective Exploration in Cooperative Multi-Agent Reinforcement Learningcited by 0
  10. 2025Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardwarecited by 0
  11. 2025Constraint-Aware Reinforcement Learning via Adaptive Action Scalingcited by 0
  12. 2026Policy Optimality Measurement for Multi-Vehicle Decision-Making: From Extrinsic Indicators to Intrinsic Qualitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Shared Information-Based Safe And Efficient Behavior Planning For Connected Autonomous Vehicles
  2. 2024OPTIMA: Optimized Policy for Intelligent Multi-Agent Systems Enables Coordination-Aware Autonomous Vehicles
  3. 2023Multi-Agent Deep Reinforcement Learning for Cooperative and Competitive Autonomous Vehicles using AutoDRIVE Ecosystem
  4. 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systems
  5. 2024Towards Safe and Robust Autonomous Vehicle Platooning: A Self-Organizing Cooperative Control Framework
  6. 2024Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving
  7. 2021Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning
  8. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning