# How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies

arXiv: https://arxiv.org/abs/2203.08409 · HTML: https://mlsanity.com/paper/2203.08409/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Placed in

- End-to-end autonomous driving / [Reinforcement learning for driving](https://mlsanity.com/driving/reinforcement-learning-for-driving/)

## Builds on (most cited first, 19 of 19)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Rainbow: Combining Improvements in Deep Reinforcement Learning](https://mlsanity.com/paper/1710.02298/) (2017)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Safe Reinforcement Learning via Shielding](https://mlsanity.com/paper/1708.08611/) (2017)
- [Reward Constrained Policy Optimization](https://mlsanity.com/paper/1805.11074/) (2018)
- [European Union regulations on algorithmic decision-making and a "right to explanation"](https://mlsanity.com/paper/1606.08813/) (2016)
- [Responsive Safety in Reinforcement Learning by PID Lagrangian Methods](https://mlsanity.com/paper/2007.03964/) (2020)
- [Safe Exploration in Continuous Action Spaces](https://mlsanity.com/paper/1801.08757/) (2018)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Conservative Safety Critics for Exploration](https://mlsanity.com/paper/2010.14497/) (2020)
- [Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving](https://mlsanity.com/paper/1905.02680/) (2019)
- [Automated Speed and Lane Change Decision Making using Deep Reinforcement Learning](https://mlsanity.com/paper/1803.10056/) (2018)
- [Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learning](https://mlsanity.com/paper/2002.02667/) (2020)
- [An Introduction to Multi-Agent Reinforcement Learning and Review of its Application to Autonomous Mobility](https://mlsanity.com/paper/2203.07676/) (2022)
- [Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections](https://mlsanity.com/paper/2006.09786/) (2020)

## Built on by (most cited first, 4 of 4)

- [Driver Dojo: A Benchmark for Generalizable Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2207.11432/) (2022)
- [Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection](https://mlsanity.com/paper/2503.19690/) (2025)
- [Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2406.13223/) (2024)
- [Constrained Reinforcement Learning Using Successor Representations](https://mlsanity.com/paper/2607.24057/) (2026)

## Abstract (arXiv)

Autonomous driving has the potential to revolutionize mobility and is hence an active area of research. In practice, the behavior of autonomous vehicles must be acceptable, i.e., efficient, safe, and interpretable. While vanilla reinforcement learning (RL) finds performant behavioral strategies, they are often unsafe and uninterpretable. Safety is introduced through Safe RL approaches, but they still mostly remain uninterpretable as the learned behaviour is jointly optimized for safety and performance without modeling them separately. Interpretable machine learning is rarely applied to RL. This paper proposes SafeDQN, which allows to make the behavior of autonomous vehicles safe and interpretable while still being efficient. SafeDQN offers an understandable, semantic trade-off between the expected risk and the utility of actions while being algorithmically transparent. We show that SafeDQN finds interpretable and safe driving policies for a variety of scenarios and demonstrate how state-of-the-art saliency techniques can help to assess both risk and utility.