# Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies

arXiv: https://arxiv.org/abs/2112.11937 · HTML: https://mlsanity.com/paper/2112.11937/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 29 of its 54 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Ray: A Distributed Framework for Emerging AI Applications](https://mlsanity.com/paper/1712.05889/) (2017)
- [DeepXplore: Automated Whitebox Testing of Deep Learning Systems](https://mlsanity.com/paper/1705.06640/) (2017)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [DeepDriving: Learning Affordance for Direct Perception in Autonomous Driving](https://mlsanity.com/paper/1505.00256/) (2015)
- [Adversarial Policies: Attacking Deep Reinforcement Learning](https://mlsanity.com/paper/1905.10615/) (2019)
- [Dealing with Non-Stationarity in Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/1906.04737/) (2019)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [Adversarial Objects Against LiDAR-Based Autonomous Driving Systems](https://mlsanity.com/paper/1907.05418/) (2019)
- [DARTS: Deceiving Autonomous Cars with Toxic Signs](https://mlsanity.com/paper/1802.06430/) (2018)
- [Multi-Agent Connected Autonomous Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1911.04175/) (2019)
- [A Systematic Review of Robustness in Deep Learning for Computer Vision: Mind the gap?](https://mlsanity.com/paper/2112.00639/) (2021)
- [Deep Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1811.11329/) (2018)
- [Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving](https://mlsanity.com/paper/1903.10654/) (2019)
- [Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation](https://mlsanity.com/paper/1908.01046/) (2019)
- [Attacking Vision-based Perception in End-to-End Autonomous Driving Models](https://mlsanity.com/paper/1910.01907/) (2019)
- [Robustifying Reinforcement Learning Agents via Action Space Adversarial Training](https://mlsanity.com/paper/2007.07176/) (2020)
- [Comparing Offline and Online Testing of Deep Neural Networks: An Autonomous Car Case Study](https://mlsanity.com/paper/1912.00805/) (2019)
- [Simulation to Scaled City: Zero-Shot Policy Transfer for Traffic Control via Autonomous Vehicles](https://mlsanity.com/paper/1812.06120/) (2018)
- [Query-based Targeted Action-Space Adversarial Policies on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/2011.07114/) (2020)
- [Robust Reinforcement Learning-based Autonomous Driving Agent for Simulation and Real World](https://mlsanity.com/paper/2009.11212/) (2020)
- [Finding Physical Adversarial Examples for Autonomous Driving with Fast and Differentiable Image Compositing](https://mlsanity.com/paper/2010.08844/) (2020)
- [Yes, we GAN: Applying Adversarial Techniques for Autonomous Driving](https://mlsanity.com/paper/1902.03442/) (2019)
- [Zero-Shot Autonomous Vehicle Policy Transfer: From Simulation to Real-World via Adversarial Learning](https://mlsanity.com/paper/1903.05252/) (2019)
- [Autonomous Driving in Reality with Reinforcement Learning and Image Translation](https://mlsanity.com/paper/1801.05299/) (2018)
- [Imminent Collision Mitigation with Reinforcement Learning and Vision](https://mlsanity.com/paper/1901.00898/) (2019)

## Built on by (most cited first, 10 of 10)

- [Robust Driving Policy Learning with Guided Meta Reinforcement Learning](https://mlsanity.com/paper/2307.10160/) (2023)
- [Toward Evaluating Robustness of Reinforcement Learning with Adversarial Policy](https://mlsanity.com/paper/2305.02605/) (2023)
- [Reinforcement Learning for Online Testing of Autonomous Driving Systems: a Replication and Extension Study](https://mlsanity.com/paper/2403.13729/) (2024)
- [CRASH: Challenging Reinforcement-Learning Based Adversarial Scenarios For Safety Hardening](https://mlsanity.com/paper/2411.16996/) (2024)
- [Sharpening the Spear: Adaptive Expert-Guided Adversarial Attack Against DRL-based Autonomous Driving Policies](https://mlsanity.com/paper/2506.18304/) (2025)
- [LLM-based Realistic Safety-Critical Driving Video Generation](https://mlsanity.com/paper/2507.01264/) (2025)
- [Suicidal Pedestrian: Generation of Safety-Critical Scenarios for Autonomous Vehicles](https://mlsanity.com/paper/2309.00249/) (2023)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Towards a Reward-Free Reinforcement Learning Framework for Vehicle Control](https://mlsanity.com/paper/2502.15262/) (2025)
- [Artificial Intelligence for Modeling and Simulation of Mixed Automated and Human Traffic](https://mlsanity.com/paper/2604.12857/) (2026)

## Abstract (arXiv)

Autonomous cars are well known for being vulnerable to adversarial attacks that can compromise the safety of the car and pose danger to other road users. To effectively defend against adversaries, it is required to not only test autonomous cars for finding driving errors but to improve the robustness of the cars to these errors. To this end, in this paper, we propose a two-step methodology for autonomous cars that consists of (i) finding failure states in autonomous cars by training the adversarial driving agent, and (ii) improving the robustness of autonomous cars by retraining them with effective adversarial inputs. Our methodology supports testing autonomous cars in a multi-agent environment, where we train and compare adversarial car policy on two custom reward functions to test the driving control decision of autonomous cars. We run experiments in a vision-based high-fidelity urban driving simulated environment. Our results show that adversarial testing can be used for finding erroneous autonomous driving behavior, followed by adversarial training for improving the robustness of deep reinforcement learning-based autonomous driving policies. We demonstrate that the autonomous cars retrained using the effective adversarial inputs noticeably increase the performance of their driving policies in terms of reduced collision and offroad steering errors.