# Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/2107.07316 · HTML: https://mlsanity.com/paper/2107.07316/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 17 of its 36 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 17 of 17)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [Algorithms for Verifying Deep Neural Networks](https://mlsanity.com/paper/1903.06758/) (2019)
- [Worst Cases Policy Gradients](https://mlsanity.com/paper/1911.03618/) (2019)
- [Safe Reinforcement Learning on Autonomous Vehicles](https://mlsanity.com/paper/1910.00399/) (2019)
- [Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learning](https://mlsanity.com/paper/2004.04450/) (2020)
- [Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environments](https://mlsanity.com/paper/1904.11483/) (2019)
- [Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learning](https://mlsanity.com/paper/1810.10469/) (2018)
- [Tackling Occlusions & Limited Sensor Range with Set-based Safety Verification](https://mlsanity.com/paper/1807.01262/) (2018)
- [Dynamic Input for Deep Reinforcement Learning in Autonomous Driving](https://mlsanity.com/paper/1907.10994/) (2019)
- [Runtime Safety Assurance Using Reinforcement Learning](https://mlsanity.com/paper/2010.10618/) (2020)
- [A Risk and Comfort Optimizing Motion Planning Scheme for Merging Scenarios](https://mlsanity.com/paper/1911.01706/) (2019)

## Built on by (most cited first, 10 of 10)

- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learning](https://mlsanity.com/paper/2203.14749/) (2022)
- [On Using Admissible Bounds for Learning Forward Search Heuristics](https://mlsanity.com/paper/2308.11905/) (2023)
- [Decentralized Multi-Robot Navigation for Autonomous Surface Vehicles with Distributional Reinforcement Learning](https://mlsanity.com/paper/2402.11799/) (2024)
- [BAP-SRL: Bayesian Adaptive Priority Safe Reinforcement Learning for Vehicle Motion Planning at Mixed Traffic Intersections](https://mlsanity.com/paper/2601.21679/) (2026)
- [Policy Evaluation in Distributional LQR](https://mlsanity.com/paper/2303.13657/) (2023)
- [Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learning](https://mlsanity.com/paper/2503.10318/) (2025)
- [Game-Theoretic Risk-Shaped Reinforcement Learning for Safe Autonomous Driving](https://mlsanity.com/paper/2510.10960/) (2025)
- [Safety Reinforced Model Predictive Control (SRMPC): Improving MPC with Reinforcement Learning for Motion Planning in Autonomous Driving](https://mlsanity.com/paper/2512.03774/) (2025)
- [Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2605.30576/) (2026)

## Abstract (arXiv)

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL can be adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent drives in average 8 seconds faster than the normal DQN policy and requires 83% less safety interference compared to the rule-based policy with slightly increasing the average crossing time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.