mlsanityData to 14 Oct 2026

Multi-Agent Vulnerability Discovery for Autonomous Driving with Hazard Arbitration Reward

arXiv 2112.06185 ↗First posted 12 Dec 2021cs.RO cs.AIcited by 1

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Discovering hazardous scenarios is crucial in testing and further improving driving policies. However, conducting efficient driving policy testing faces two key challenges. On the one hand, the probability of naturally encountering hazardous scenarios is low when testing a well-trained autonomous driving strategy. Thus, discovering these scenarios by purely real-world road testing is extremely costly. On the other hand, a proper determination of accident responsibility is necessary for this task. Collecting scenarios with wrong-attributed responsibilities will lead to an overly conservative autonomous driving strategy. To be more specific, we aim to discover hazardous scenarios that are autonomous-vehicle responsible (AV-responsible), i.e., the vulnerabilities of the under-test driving policy. To this end, this work proposes a Safety Test framework by finding Av-Responsible Scenarios (STARS) based on multi-agent reinforcement learning. STARS guides other traffic participants to produce Av-Responsible Scenarios and make the under-test driving policy misbehave via introducing Hazard Arbitration Reward (HAR). HAR enables our framework to discover diverse, complex, and AV-responsible hazardous scenarios. Experimental results against four different driving policies in three environments demonstrate that STARS can effectively discover AV-responsible hazardous scenarios. These scenarios indeed correspond to the vulnerabilities of the under-test driving policies, thus are meaningful for their further improvements.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 16 of its 23 references that are in the map. First posted on arXiv 12 Dec 2021.

Builds on

16 of 23 references.

References · 16

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2015Continuous control with deep reinforcement learningcited by 3,407
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  5. 2015Dueling Network Architectures for Deep Reinforcement Learningcited by 785
  6. 2017AirSim: High-Fidelity Visual and Physical Simulation for Autonomous Vehiclescited by 523
  7. 2019Adversarial Policies: Attacking Deep Reinforcement Learningcited by 214
  8. 2018The ApolloScape Open Dataset for Autonomous Driving and its Applicationcited by 190
  9. 2017Tactics of Adversarial Attack on Deep Reinforcement Learning Agentscited by 166
  10. 2017On a Formal Model of Safe and Scalable Self-driving Carscited by 131
  11. 2020SMARTS: Scalable Multi-Agent Reinforcement Learning Training School for Autonomous Drivingcited by 121
  12. 2019Certified Adversarial Robustness for Deep Reinforcement Learningcited by 47
4 more
  1. 2019Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Drivingcited by 36
  2. 2020Training Adversarial Agents to Exploit Weaknesses in Deep Control Policiescited by 24
  3. 2018Adversarial Reinforcement Learning Framework for Benchmarking Collision Avoidance Mechanisms in Autonomous Vehiclescited by 8
  4. 2020Driving-Policy Adaptive Safeguard for Autonomous Vehicles Using Reinforcement Learningcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Rethinking Robustness Assessment: Adversarial Attacks on Learning-based Quadrupedal Locomotion Controllerscited by 14

Similar, not cited

Close in wording, no citation either way

  1. 2024Generating Critical Scenarios for Testing Automated Driving Systems
  2. 2025AED: Automatic Discovery of Effective and Diverse Vulnerabilities for Autonomous Driving Policy with Large Language Models
  3. 2025MARL-OT: Multi-Agent Reinforcement Learning Guided Online Fuzzing to Detect Safety Violation in Autonomous Driving Systems
  4. 2022Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios
  5. 2025Reinforcement Learning for Testing Interdependent Requirements in Autonomous Vehicles: An Empirical Study
  6. 2025ROAD: Responsibility-Oriented Reward Design for Reinforcement Learning in Autonomous Driving
  7. 2026Learning Responsibility-Attributed Adversarial Scenarios for Testing Autonomous Vehicles
  8. 2025Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving