# Adaptive Testing Environment Generation for Connected and Automated Vehicles with Dense Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.19275 · HTML: https://mlsanity.com/paper/2402.19275/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 63 references that are in the map.
- First posted on arXiv 29 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 20 of 20)

- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Generating Useful Accident-Prone Driving Scenarios via a Learned Traffic Prior](https://mlsanity.com/paper/2112.05077/) (2021)
- [AdvSim: Generating Safety-Critical Scenarios for Self-Driving Vehicles](https://mlsanity.com/paper/2101.06549/) (2021)
- [Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation](https://mlsanity.com/paper/1811.00145/) (2018)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- ["Curse of rarity" for autonomous vehicles](https://mlsanity.com/paper/2207.02749/) (2022)
- [Scenarios for Development, Test and Validation of Automated Vehicles](https://mlsanity.com/paper/1801.08598/) (2018)
- [Accelerated Evaluation of Automated Vehicles Safety in Lane Change Scenarios Based on Importance Sampling Techniques](https://mlsanity.com/paper/1605.04965/) (2016)
- [Neural Bridge Sampling for Evaluating Safety-Critical Autonomous Systems](https://mlsanity.com/paper/2008.10581/) (2020)
- [Adaptive Stress Testing with Reward Augmentation for Autonomous Vehicle Validation](https://mlsanity.com/paper/1908.01046/) (2019)
- [Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles](https://mlsanity.com/paper/2102.03483/) (2021)
- [Testing Scenario Library Generation for Connected and Automated Vehicles, Part I: Methodology](https://mlsanity.com/paper/1905.03419/) (2019)
- [Efficient Black-box Assessment of Autonomous Vehicle Safety](https://mlsanity.com/paper/1912.03618/) (2019)
- [Requirements-driven Test Generation for Autonomous Vehicles with Machine Learning Components](https://mlsanity.com/paper/1908.01094/) (2019)
- [Testing Scenario Library Generation for Connected and Automated Vehicles, Part II: Case Studies](https://mlsanity.com/paper/1905.03428/) (2019)
- [Adaptive Safety Evaluation for Connected and Automated Vehicles with Sparse Control Variates](https://mlsanity.com/paper/2212.00517/) (2022)
- [An adaptive multi-fidelity sampling framework for safety analysis of connected and automated vehicles](https://mlsanity.com/paper/2210.14114/) (2022)
- [Towards Guaranteed Safety Assurance of Automated Driving Systems with Scenario Sampling: An Invariant Set Perspective (Extended Version)](https://mlsanity.com/paper/2104.09595/) (2021)
- [Adaptive Testing for Connected and Automated Vehicles with Sparse Control Variates in Overtaking Scenarios](https://mlsanity.com/paper/2207.09259/) (2022)
- [Few-Shot Scenario Testing for Autonomous Vehicles Based on Neighborhood Coverage and Similarity](https://mlsanity.com/paper/2402.01795/) (2024)

## Built on by (most cited first, 2 of 2)

- [Towards provable probabilistic safety for scalable embodied AI systems](https://mlsanity.com/paper/2506.05171/) (2025)
- [Open-Source Autonomous Driving System Analysis and Multi-Disciplinary Hardware-in-the-Loop Research Paradigm with Reinforcement-Learning Testing and Large Language Models](https://mlsanity.com/paper/2608.30179/) (2026)

## Abstract (arXiv)

The assessment of safety performance plays a pivotal role in the development and deployment of connected and automated vehicles (CAVs). A common approach involves designing testing scenarios based on prior knowledge of CAVs (e.g., surrogate models), conducting tests in these scenarios, and subsequently evaluating CAVs' safety performances. However, substantial differences between CAVs and the prior knowledge can significantly diminish the evaluation efficiency. In response to this issue, existing studies predominantly concentrate on the adaptive design of testing scenarios during the CAV testing process. Yet, these methods have limitations in their applicability to high-dimensional scenarios. To overcome this challenge, we develop an adaptive testing environment that bolsters evaluation robustness by incorporating multiple surrogate models and optimizing the combination coefficients of these surrogate models to enhance evaluation efficiency. We formulate the optimization problem as a regression task utilizing quadratic programming. To efficiently obtain the regression target via reinforcement learning, we propose the dense reinforcement learning method and devise a new adaptive policy with high sample efficiency. Essentially, our approach centers on learning the values of critical scenes displaying substantial surrogate-to-real gaps. The effectiveness of our method is validated in high-dimensional overtaking scenarios, demonstrating that our approach achieves notable evaluation efficiency.