# Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimation

arXiv: https://arxiv.org/abs/2004.10439 · HTML: https://mlsanity.com/paper/2004.10439/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Apr 2020.

## Built on by (most cited first, 14 of 14)

- [Q-attention: Enabling Efficient Learning for Vision-based Robotic Manipulation](https://mlsanity.com/paper/2105.14829/) (2021)
- [Transformer Uncertainty Estimation with Hierarchical Stochastic Attention](https://mlsanity.com/paper/2112.13776/) (2021)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Driving](https://mlsanity.com/paper/2501.08096/) (2025)
- [Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2305.07487/) (2023)
- [Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priors](https://mlsanity.com/paper/2406.19768/) (2024)
- [Failure Detection for Motion Prediction of Autonomous Driving: An Uncertainty Perspective](https://mlsanity.com/paper/2301.04421/) (2023)
- [Tactical Decision Making for Autonomous Trucks by Deep Reinforcement Learning with Total Cost of Operation Based Reward](https://mlsanity.com/paper/2403.06524/) (2024)
- [Pedestrian-Aware LLM-Driven Behavioral Planning for Autonomous Vehicles](https://mlsanity.com/paper/2605.16858/) (2026)
- [Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios](https://mlsanity.com/paper/2204.05196/) (2022)
- [Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning](https://mlsanity.com/paper/2204.06509/) (2022)
- [MSVIPER: Improved Policy Distillation for Reinforcement-Learning-Based Robot Navigation](https://mlsanity.com/paper/2209.09079/) (2022)
- [Preference-Conditioned Multi-Objective Reinforcement Learning for Tactical Decision Making in Heavy-Duty Vehicles](https://mlsanity.com/paper/2601.18783/) (2026)
- [Multi-Agent Reinforcement Learning for Safe Autonomous Driving Under Pedestrian Behavioral Uncertainty](https://mlsanity.com/paper/2605.20255/) (2026)

## Abstract (arXiv)

Reinforcement learning (RL) can be used to create a tactical decision-making agent for autonomous driving. However, previous approaches only output decisions and do not provide information about the agent's confidence in the recommended actions. This paper investigates how a Bayesian RL technique, based on an ensemble of neural networks with additional randomized prior functions (RPF), can be used to estimate the uncertainty of decisions in autonomous driving. A method for classifying whether or not an action should be considered safe is also introduced. The performance of the ensemble RPF method is evaluated by training an agent on a highway driving scenario. It is shown that the trained agent can estimate the uncertainty of its decisions and indicate an unacceptable level when the agent faces a situation that is far from the training distribution. Furthermore, within the training distribution, the ensemble RPF agent outperforms a standard Deep Q-Network agent. In this study, the estimated uncertainty is used to choose safe actions in unknown situations. However, the uncertainty information could also be used to identify situations that should be added to the training process.