# Robust and Safe Multi-Agent Reinforcement Learning with Communication for Autonomous Vehicles: From Simulation to Hardware

arXiv: https://arxiv.org/abs/2506.00982 · HTML: https://mlsanity.com/paper/2506.00982/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Cooperative and V2X driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 1 Jun 2025.

## Placed in

- End-to-end autonomous driving / [Cooperative and V2X driving](https://mlsanity.com/driving/cooperative-and-v2x-driving/)

## Builds on (most cited first, 25 of 25)

- [Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real World](https://mlsanity.com/paper/1703.06907/) (2017)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes](https://mlsanity.com/paper/2408.03539/) (2024)
- [Active Domain Randomization](https://mlsanity.com/paper/1904.04762/) (2019)
- [V2X-Sim: Multi-Agent Collaborative Perception Dataset and Benchmark for Autonomous Driving](https://mlsanity.com/paper/2202.08449/) (2022)
- [Reconciling Reality through Simulation: A Real-to-Sim-to-Real Approach for Robust Manipulation](https://mlsanity.com/paper/2403.03949/) (2024)
- [Ultra Fast Structure-aware Deep Lane Detection](https://mlsanity.com/paper/2004.11757/) (2020)
- [Safe Multi-Agent Reinforcement Learning via Shielding](https://mlsanity.com/paper/2101.11196/) (2021)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [Efficient Adversarial Training without Attacking: Worst-Case-Aware Robust Reinforcement Learning](https://mlsanity.com/paper/2210.05927/) (2022)
- [Robust Multi-Agent Reinforcement Learning with State Uncertainty](https://mlsanity.com/paper/2307.16212/) (2023)
- [What is the Solution for State-Adversarial Multi-Agent Reinforcement Learning?](https://mlsanity.com/paper/2212.02705/) (2022)
- [Spatial-Temporal-Aware Safe Multi-Agent Reinforcement Learning of Connected Autonomous Vehicles in Challenging Scenarios](https://mlsanity.com/paper/2210.02300/) (2022)
- [Learning Multi-Agent Loco-Manipulation for Long-Horizon Quadrupedal Pushing](https://mlsanity.com/paper/2411.07104/) (2024)
- [A Multi-Agent Reinforcement Learning Approach For Safe and Efficient Behavior Planning Of Connected Autonomous Vehicles](https://mlsanity.com/paper/2003.04371/) (2020)
- [Stable and Efficient Shapley Value-Based Reward Reallocation for Multi-Agent Reinforcement Learning of Autonomous Vehicles](https://mlsanity.com/paper/2203.06333/) (2022)
- [A Fleet of Miniature Cars for Experiments in Cooperative Driving](https://mlsanity.com/paper/1902.06133/) (2019)
- [Safe Multi-Agent Reinforcement Learning through Decentralized Multiple Control Barrier Functions](https://mlsanity.com/paper/2103.12553/) (2021)
- [The Cambridge RoboMaster: An Agile Multi-Robot Research Platform](https://mlsanity.com/paper/2405.02198/) (2024)
- [Multi-Agent Reinforcement Learning Guided by Signal Temporal Logic Specifications](https://mlsanity.com/paper/2306.06808/) (2023)
- [Reinforcement Learning-based Receding Horizon Control using Adaptive Control Barrier Functions for Safety-Critical Systems](https://mlsanity.com/paper/2403.17338/) (2024)
- [Safety Guaranteed Robust Multi-Agent Reinforcement Learning with Hierarchical Control for Connected and Automated Vehicles](https://mlsanity.com/paper/2309.11057/) (2023)
- [A Survey on Small-Scale Testbeds for Connected and Automated Vehicles and Robot Swarms](https://mlsanity.com/paper/2408.14199/) (2024)
- [Learning Adaptive Safety for Multi-Agent Systems](https://mlsanity.com/paper/2309.10657/) (2023)

## Abstract (arXiv)

Deep multi-agent reinforcement learning (MARL) has been demonstrated effectively in simulations for multi-robot problems. For autonomous vehicles, the development of vehicle-to-vehicle (V2V) communication technologies provide opportunities to further enhance system safety. However, zero-shot transfer of simulator-trained MARL policies to dynamic hardware systems remains challenging, and how to leverage communication and shared information for MARL has limited demonstrations on hardware. This problem is challenged by discrepancies between simulated and physical states, system state and model uncertainties, practical shared information design, and the need for safety guarantees in both simulation and hardware. This paper designs RSR-RSMARL, a novel Robust and Safe MARL framework that supports Real-Sim-Real (RSR) policy adaptation for multi-agent systems with communication among agents, with both simulation and hardware demonstrations. RSR-RSMARL leverages state (includes shared state information among agents) and action representations considering real system complexities for MARL formulation. The MARL policy is trained with robust MARL algorithm to enable zero-shot transfer to hardware considering the sim-to-real gap. A safety shield module using Control Barrier Functions (CBFs) provides safety guarantee for each individual agent. Experimental results on 1/10th-scale autonomous vehicles with V2V communication demonstrate the ability of RSR-RSMARL framework to enhance driving safety and coordination across multiple configurations. These findings emphasize the importance of jointly designing robust policy representations and modular safety architectures to enable scalable, generalizable RSR transfer in multi-agent autonomy.