# Failure-Scenario Maker for Rule-Based Agent using Multi-agent Adversarial Reinforcement Learning and its Application to Autonomous Driving

arXiv: https://arxiv.org/abs/1903.10654 · HTML: https://mlsanity.com/paper/1903.10654/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Mar 2019.

## Built on by (most cited first, 30 of 36)

- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehicles](https://mlsanity.com/paper/2405.14062/) (2024)
- [CAT: Closed-loop Adversarial Training for Safe End-to-End Driving](https://mlsanity.com/paper/2310.12432/) (2023)
- [Vehicle trajectory prediction works, but not everywhere](https://mlsanity.com/paper/2112.03909/) (2021)
- [Multi-Agent Coordination across Diverse Applications: A Survey](https://mlsanity.com/paper/2502.14743/) (2025)
- [The Integration of Prediction and Planning in Deep Learning Automated Driving Systems: A Review](https://mlsanity.com/paper/2308.05731/) (2023)
- [A Survey on Scenario-Based Testing for Automated Driving Systems in High-Fidelity Simulation](https://mlsanity.com/paper/2112.00964/) (2021)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Safety-Critical Scenario Generation Via Reinforcement Learning Based Editing](https://mlsanity.com/paper/2306.14131/) (2023)
- [Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priors](https://mlsanity.com/paper/2408.03200/) (2024)
- [Learning to Drive via Asymmetric Self-Play](https://mlsanity.com/paper/2409.18218/) (2024)
- [Adversarial Deep Reinforcement Learning for Improving the Robustness of Multi-agent Autonomous Driving Policies](https://mlsanity.com/paper/2112.11937/) (2021)
- [Adv3D: Generating Safety-Critical 3D Objects through Closed-Loop Simulation](https://mlsanity.com/paper/2311.01446/) (2023)
- [Steerable Adversarial Scenario Generation through Test-Time Preference Alignment](https://mlsanity.com/paper/2509.20102/) (2025)
- [(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning](https://mlsanity.com/paper/2302.13726/) (2023)
- [From Model-Based to Data-Driven Simulation: Challenges and Trends in Autonomous Driving](https://mlsanity.com/paper/2305.13960/) (2023)
- [Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving](https://mlsanity.com/paper/2309.14235/) (2023)
- [Adversarial and Reactive Traffic Entities for Behavior-Realistic Driving Simulation: A Review](https://mlsanity.com/paper/2409.14196/) (2024)
- [ADV-0: Closed-Loop Min-Max Adversarial Training for Long-Tail Robustness in Autonomous Driving](https://mlsanity.com/paper/2603.15221/) (2026)
- [ECSAS: Exploring Critical Scenarios from Action Sequence in Autonomous Driving](https://mlsanity.com/paper/2209.10078/) (2022)
- [A Bibliometric Analysis and Review on Reinforcement Learning for Transportation Applications](https://mlsanity.com/paper/2210.14524/) (2022)
- [Learning to Model Diverse Driving Behaviors in Highly Interactive Autonomous Driving Scenarios with Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2402.13481/) (2024)
- [AED: Automatic Discovery of Effective and Diverse Vulnerabilities for Autonomous Driving Policy with Large Language Models](https://mlsanity.com/paper/2503.20804/) (2025)
- [Multi-Agent Vulnerability Discovery for Autonomous Driving with Hazard Arbitration Reward](https://mlsanity.com/paper/2112.06185/) (2021)
- [Towards Benchmarking and Assessing the Safety and Robustness of Autonomous Driving on Safety-critical Scenarios](https://mlsanity.com/paper/2503.23708/) (2025)
- [Embodied Intelligence: The Key to Unblocking Generalized Artificial Intelligence](https://mlsanity.com/paper/2505.06897/) (2025)
- [An Evolving Scenario Generation Method based on Dual-modal Driver Model Trained by Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2508.02027/) (2025)
- [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023)
- [ReMAV: Reward Modeling of Autonomous Vehicles for Finding Likely Failure Events](https://mlsanity.com/paper/2308.14550/) (2023)
- [Acceleration method for generating perception failure scenarios based on editing Markov process](https://mlsanity.com/paper/2407.00980/) (2024)

## Abstract (arXiv)

We examine the problem of adversarial reinforcement learning for multi-agent domains including a rule-based agent. Rule-based algorithms are required in safety-critical applications for them to work properly in a wide range of situations. Hence, every effort is made to find failure scenarios during the development phase. However, as the software becomes complicated, finding failure cases becomes difficult. Especially in multi-agent domains, such as autonomous driving environments, it is much harder to find useful failure scenarios that help us improve the algorithm. We propose a method for efficiently finding failure scenarios; this method trains the adversarial agents using multi-agent reinforcement learning such that the tested rule-based agent fails. We demonstrate the effectiveness of our proposed method using a simple environment and autonomous driving simulator.