# Dynamically Expanding Capacity of Autonomous Driving with Near-Miss Focused Training Framework

arXiv: https://arxiv.org/abs/2406.02865 · HTML: https://mlsanity.com/paper/2406.02865/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Adversarial scenarios and safety testing, one of 9 directions in End-to-end autonomous driving.
- Builds on 13 of its 18 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Placed in

- End-to-end autonomous driving / [Adversarial scenarios and safety testing](https://mlsanity.com/driving/adversarial-scenarios-and-safety-testing/)

## Builds on (most cited first, 13 of 13)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Robust Adversarial Reinforcement Learning](https://mlsanity.com/paper/1703.02702/) (2017)
- [A Survey on Safety-Critical Driving Scenario Generation -- A Methodological Perspective](https://mlsanity.com/paper/2202.02215/) (2022)
- [Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles](https://mlsanity.com/paper/2102.03483/) (2021)
- [Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology](https://mlsanity.com/paper/1905.03419/) (2019)
- [Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies](https://mlsanity.com/paper/2002.12078/) (2020)
- [CMTS: Conditional Multiple Trajectory Synthesizer for Generating Safety-critical Driving Scenarios](https://mlsanity.com/paper/1910.00099/) (2019)
- [DR2L: Surfacing Corner Cases to Robustify Autonomous Driving via Domain Randomization Reinforcement Learning](https://mlsanity.com/paper/2107.11762/) (2021)
- [(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning](https://mlsanity.com/paper/2302.13726/) (2023)
- [Continual Driving Policy Optimization with Closed-Loop Individualized Curricula](https://mlsanity.com/paper/2309.14209/) (2023)
- [Stackelberg Driver Model for Continual Policy Improvement in Scenario-Based Closed-Loop Autonomous Driving](https://mlsanity.com/paper/2309.14235/) (2023)
- [Rapidly-exploring Random Trees-based Test Generation for Autonomous Vehicles](https://mlsanity.com/paper/1903.10629/) (2019)

## Abstract (arXiv)

The long-tail distribution of real driving data poses challenges for training and testing autonomous vehicles (AV), where rare yet crucial safety-critical scenarios are infrequent. And virtual simulation offers a low-cost and efficient solution. This paper proposes a near-miss focused training framework for AV. Utilizing the driving scenario information provided by sensors in the simulator, we design novel reward functions, which enable background vehicles (BV) to generate near-miss scenarios and ensure gradients exist not only in collision-free scenes but also in collision scenarios. And then leveraging the Robust Adversarial Reinforcement Learning (RARL) framework for simultaneous training of AV and BV to gradually enhance AV and BV capabilities, as well as generating near-miss scenarios tailored to different levels of AV capabilities. Results from three testing strategies indicate that the proposed method generates scenarios closer to near-miss, thus enhancing the capabilities of both AVs and BVs throughout training.