# Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge

arXiv: https://arxiv.org/abs/2103.10245 · HTML: https://mlsanity.com/paper/2103.10245/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 20 of its 59 references that are in the map.
- First posted on arXiv 16 Mar 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 20 of 20)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Dueling Network Architectures for Deep Reinforcement Learning](https://mlsanity.com/paper/1511.06581/) (2015)
- [ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst](https://mlsanity.com/paper/1812.03079/) (2018)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Adversarial Examples for Semantic Segmentation and Object Detection](https://mlsanity.com/paper/1703.08603/) (2017)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Learning to Collide: An Adaptive Safety-Critical Scenarios Generating Method](https://mlsanity.com/paper/2003.01197/) (2020)
- [Adaptive Stress Testing for Autonomous Vehicles](https://mlsanity.com/paper/1902.01909/) (2019)
- [Conditional Affordance Learning for Driving in Urban Environments](https://mlsanity.com/paper/1806.06498/) (2018)
- [Corner Case Generation and Analysis for Safety Assessment of Autonomous Vehicles](https://mlsanity.com/paper/2102.03483/) (2021)
- [CMetric: A Driving Behavior Measure Using Centrality Functions](https://mlsanity.com/paper/2003.04424/) (2020)
- [Review of The Book of Why: The New Science of Cause and Effect](https://mlsanity.com/paper/2003.11635/) (2020)
- [Expert Level control of Ramp Metering based on Multi-task Deep Reinforcement Learning](https://mlsanity.com/paper/1701.08832/) (2017)
- [Approximate Robust Control of Uncertain Dynamical Systems](https://mlsanity.com/paper/1903.00220/) (2019)
- [Practical Open-Loop Optimistic Planning](https://mlsanity.com/paper/1904.04700/) (2019)
- [Distributionally Consistent Simulation of Naturalistic Driving Environment for Autonomous Vehicle Testing](https://mlsanity.com/paper/2101.02828/) (2021)
- [Interval Prediction for Continuous-Time Systems with Parametric Uncertainties](https://mlsanity.com/paper/1904.04727/) (2019)

## Built on by (most cited first, 4 of 4)

- [From Words to Collisions: LLM-Guided Evaluation and Adversarial Generation of Safety-Critical Driving Scenarios](https://mlsanity.com/paper/2502.02145/) (2025)
- [(Re)2H2O: Autonomous Driving Scenario Generation via Reversely Regularized Hybrid Offline-and-Online Reinforcement Learning](https://mlsanity.com/paper/2302.13726/) (2023)
- [Exploring the trade off between human driving imitation and safety for traffic simulation](https://mlsanity.com/paper/2208.04803/) (2022)
- [Controllable risk scenario generation from human crash data for autonomous vehicle testing](https://mlsanity.com/paper/2512.07874/) (2025)

## Abstract (arXiv)

Simulation environments are good for learning different driving tasks like lane changing, parking or handling intersections etc. in an abstract manner. However, these simulation environments often restrict themselves to operate under conservative interaction behavior amongst different vehicles. But, as we know, real driving tasks often involve very high risk scenarios where other drivers often don't behave in the expected sense. There can be many reasons for this behavior like being tired or inexperienced. The simulation environment doesn't take this information into account while training the navigation agent. Therefore, in this study we especially focus on systematically creating these risk prone scenarios with heavy traffic and unexpected random behavior for creating better model-free learning agents. We generate multiple autonomous driving scenarios by creating new custom Markov Decision Process (MDP) environment iterations in the highway-env simulation package. The behavior policy is learnt by agents trained with the help from deep reinforcement learning models. Our behavior policy is deliberated to handle collisions and risky randomized driver behavior. We train model free learning agents with supplement information of risk prone driving scenarios and compare their performance with baseline agents. Finally, we casually measure the impact of adding these perturbations in the training process to precisely account for the performance improvement obtained from utilizing the learnings from these scenarios.