mlsanityData to 7 Oct 2026

Minimizing Safety Interference for Safe and Comfortable Automated Driving with Distributional Reinforcement Learning

arXiv 2107.07316 ↗First posted 15 Jul 2021cs.RO cs.AIcited by 8

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Despite recent advances in reinforcement learning (RL), its application in safety critical domains like autonomous vehicles is still challenging. Although punishing RL agents for risky situations can help to learn safe policies, it may also lead to highly conservative behavior. In this paper, we propose a distributional RL framework in order to learn adaptive policies that can tune their level of conservativity at run-time based on the desired comfort and utility. Using a proactive safety verification approach, the proposed framework can guarantee that actions generated from RL are fail-safe according to the worst-case assumptions. Concurrently, the policy is encouraged to minimize safety interference and generate more comfortable behavior. We trained and evaluated the proposed approach and baseline policies using a high level simulator with a variety of randomized scenarios including several corner cases which rarely happen in reality but are very crucial. In light of our experiments, the behavior of policies learned using distributional RL can be adaptive at run-time and robust to the environment uncertainty. Quantitatively, the learned distributional RL agent drives in average 8 seconds faster than the normal DQN policy and requires 83% less safety interference compared to the rule-based policy with slightly increasing the average crossing time. We also study sensitivity of the learned policy in environments with higher perception noise and show that our algorithm learns policies that can still drive reliable when the perception noise is two times higher than the training configuration for automated merging and crossing at occluded intersections.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 17 of its 36 references that are in the map. First posted on arXiv 15 Jul 2021.

Builds on

17 of 36 references.

References · 17

  1. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  2. 2017Deep Setscited by 1,394
  3. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  4. 2017Distributional Reinforcement Learning with Quantile Regressioncited by 464
  5. 2017Safe Reinforcement Learning via Shieldingcited by 427
  6. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 362
  7. 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
  8. 2019Algorithms for Verifying Deep Neural Networkscited by 221
  9. 2019Worst Cases Policy Gradientscited by 52
  10. 2019Safe Reinforcement Learning on Autonomous Vehiclescited by 48
  11. 2020Risk-Aware High-level Decisions for Automated Driving at Occluded Intersections with Reinforcement Learningcited by 19
  12. 2019Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environmentscited by 18
5 more
  1. 2018Learning Negotiating Behavior Between Cars in Intersections using Deep Q-Learningcited by 13
  2. 2018Tackling Occlusions & Limited Sensor Range with Set-based Safety Verificationcited by 11
  3. 2019Dynamic Input for Deep Reinforcement Learning in Autonomous Drivingcited by 11
  4. 2020Runtime Safety Assurance Using Reinforcement Learningcited by 8
  5. 2019A Risk and Comfort Optimizing Motion Planning Scheme for Merging Scenarioscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 2; 2024: 0; 2025: 3; 2026: 1.

8 papers.

  1. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 169
  2. 2022Adaptive Risk-Tendency: Nano Drone Navigation in Cluttered Environments with Distributional Reinforcement Learningcited by 5
  3. 2023On Using Admissible Bounds for Learning Forward Search Heuristicscited by 5
  4. 2023Policy Evaluation in Distributional LQRcited by 0
  5. 2025Enhance Exploration in Safe Reinforcement Learning with Contrastive Representation Learningcited by 0
  6. 2025Game-Theoretic Risk-Shaped Reinforcement Learning for Safe Autonomous Drivingcited by 0
  7. 2025Safety Reinforced Model Predictive Control (SRMPC): Improving MPC with Reinforcement Learning for Motion Planning in Autonomous Drivingcited by 0
  8. 2026Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022SafeRL-Kit: Evaluating Efficient Reinforcement Learning Methods for Safe Autonomous Driving
  2. 2024Long and Short-Term Constraints Driven Safe Reinforcement Learning for Autonomous Driving
  3. 2021Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning
  4. 2025Uncertainty-Aware Safety-Critical Decision and Control for Autonomous Vehicles at Unsignalized Intersections
  5. 2025Balancing Progress and Safety: A Novel Risk-Aware Objective for RL in Autonomous Driving
  6. 2026A Risk-Sensitive and Uncertainty-Aware Decision-Making and Control Framework for Safe and Robust Autonomous Driving
  7. 2025Ethics-Aware Safe Reinforcement Learning for Rare-Event Risk Control in Interactive Urban Driving
  8. 2024Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning