# Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning

arXiv: https://arxiv.org/abs/2102.03119 · HTML: https://mlsanity.com/paper/2102.03119/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 5 Feb 2021.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 15 of 15)

- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [Distributional Reinforcement Learning with Quantile Regression](https://mlsanity.com/paper/1710.10044/) (2017)
- [Implicit Quantile Networks for Distributional Reinforcement Learning](https://mlsanity.com/paper/1806.06923/) (2018)
- [Safe, Multi-Agent, Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/1610.03295/) (2016)
- [How Should a Robot Assess Risk? Towards an Axiomatic Theory of Risk in Robotics](https://mlsanity.com/paper/1710.11040/) (2017)
- [Reinforcement Learning with Probabilistic Guarantees for Autonomous Driving](https://mlsanity.com/paper/1904.07189/) (2019)
- [Belief and Truth in Hypothesised Behaviours](https://mlsanity.com/paper/1507.07688/) (2015)
- [Navigating Occluded Intersections with Autonomous Vehicles using Deep Reinforcement Learning](https://mlsanity.com/paper/1705.01196/) (2017)
- [Combining Neural Networks and Tree Search for Task and Motion Planning in Challenging Environments](https://mlsanity.com/paper/1703.07887/) (2017)
- [Belief State Planning for Autonomously Navigating Urban Intersections](https://mlsanity.com/paper/1704.04322/) (2017)
- [Decentralized Cooperative Planning for Automated Vehicles with Hierarchical Monte Carlo Tree Search](https://mlsanity.com/paper/1807.09530/) (2018)
- [Adaptive Behavior Generation for Autonomous Driving using Deep Reinforcement Learning with Compact Semantic States](https://mlsanity.com/paper/1809.03214/) (2018)
- [Decentralized Cooperative Planning for Automated Vehicles with Continuous Monte Carlo Tree Search](https://mlsanity.com/paper/1809.03200/) (2018)

## Built on by (most cited first, 13 of 13)

- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Decision-Making Technology for Autonomous Vehicles Learning-Based Methods, Applications and Future Outlook](https://mlsanity.com/paper/2107.01110/) (2021)
- [Learning Risk-Aware Quadrupedal Locomotion using Distributional Reinforcement Learning](https://mlsanity.com/paper/2309.14246/) (2023)
- [Beyond Average Return in Markov Decision Processes](https://mlsanity.com/paper/2310.20266/) (2023)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments](https://mlsanity.com/paper/2201.12776/) (2022)
- [FineFT: Efficient and Risk-Aware Ensemble Reinforcement Learning for Futures Trading](https://mlsanity.com/paper/2512.23773/) (2025)
- [Cooperative Trajectory Planning in Uncertain Environments with Monte Carlo Tree Search and Risk Metrics](https://mlsanity.com/paper/2203.04452/) (2022)
- [Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning](https://mlsanity.com/paper/2204.06509/) (2022)
- [DRL-ORA: Distributional Reinforcement Learning with Online Risk Adaption](https://mlsanity.com/paper/2310.05179/) (2023)
- [Distributional Active Inference](https://mlsanity.com/paper/2601.20985/) (2026)
- [Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2605.30576/) (2026)
- [A Noise-Robust Elicit-to-Optimize Framework for Distortion Riskmetrics via Inverse Reinforcement Learning](https://mlsanity.com/paper/2607.14373/) (2026)

## Abstract (arXiv)

For highly automated driving above SAE level~3, behavior generation algorithms must reliably consider the inherent uncertainties of the traffic environment, e.g. arising from the variety of human driving styles. Such uncertainties can generate ambiguous decisions, requiring the algorithm to appropriately balance low-probability hazardous events, e.g. collisions, and high-probability beneficial events, e.g. quickly crossing the intersection. State-of-the-art behavior generation algorithms lack a distributional treatment of decision outcome. This impedes a proper risk evaluation in ambiguous situations, often encouraging either unsafe or conservative behavior. Thus, we propose a two-step approach for risk-sensitive behavior generation combining offline distribution learning with online risk assessment. Specifically, we first learn an optimal policy in an uncertain environment with Deep Distributional Reinforcement Learning. During execution, the optimal risk-sensitive action is selected by applying established risk criteria, such as the Conditional Value at Risk, to the learned state-action return distributions. In intersection crossing scenarios, we evaluate different risk criteria and demonstrate that our approach increases safety, while maintaining an active driving style. Our approach shall encourage further studies about the benefits of risk-sensitive approaches for self-driving vehicles.