# Reinforcement Learning with Uncertainty Estimation for Tactical Decision-Making in Intersections

arXiv: https://arxiv.org/abs/2006.09786 · HTML: https://mlsanity.com/paper/2006.09786/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2020.

## Built on by (most cited first, 15 of 15)

- [Uncertainty Weighted Actor-Critic for Offline Reinforcement Learning](https://mlsanity.com/paper/2105.08140/) (2021)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [State Dropout-Based Curriculum Reinforcement Learning for Self-Driving at Unsignalized Intersections](https://mlsanity.com/paper/2207.04361/) (2022)
- [Decision-Making Technology for Autonomous Vehicles Learning-Based Methods, Applications and Future Outlook](https://mlsanity.com/paper/2107.01110/) (2021)
- [Autonomous Driving at Unsignalized Intersections: A Review of Decision-Making Challenges and Reinforcement Learning-Based Solutions](https://mlsanity.com/paper/2409.13144/) (2024)
- [Identify, Estimate and Bound the Uncertainty of Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2305.07487/) (2023)
- [Contextualized Hybrid Ensemble Q-learning: Learning Fast with Control Priors](https://mlsanity.com/paper/2406.19768/) (2024)
- [How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies](https://mlsanity.com/paper/2203.08409/) (2022)
- [Influence of uncertainty estimation techniques on false-positive reduction in liver lesion detection](https://mlsanity.com/paper/2206.10911/) (2022)
- [Graph Convolution-Based Deep Reinforcement Learning for Multi-Agent Decision-Making in Mixed Traffic Environments](https://mlsanity.com/paper/2201.12776/) (2022)
- [Grid-Mapping Pseudo-Count Constraint for Offline Reinforcement Learning](https://mlsanity.com/paper/2404.02545/) (2024)
- [Stepping Out of the Shadows: Reinforcement Learning in Shadow Mode](https://mlsanity.com/paper/2410.23419/) (2024)
- [Evaluating Scenario-based Decision-making for Interactive Autonomous Driving Using Rational Criteria: A Survey](https://mlsanity.com/paper/2501.01886/) (2025)
- [GITSR: Graph Interaction Transformer-based Scene Representation for Multi Vehicle Collaborative Decision-making](https://mlsanity.com/paper/2411.01608/) (2024)
- [UAMDP: Uncertainty-Aware Markov Decision Process for Risk-Constrained Reinforcement Learning from Probabilistic Forecasts](https://mlsanity.com/paper/2510.08226/) (2025)

## Abstract (arXiv)

This paper investigates how a Bayesian reinforcement learning method can be used to create a tactical decision-making agent for autonomous driving in an intersection scenario, where the agent can estimate the confidence of its recommended actions. An ensemble of neural networks, with additional randomized prior functions (RPF), are trained by using a bootstrapped experience replay memory. The coefficient of variation in the estimated Q-values of the ensemble members is used to approximate the uncertainty, and a criterion that determines if the agent is sufficiently confident to make a particular decision is introduced. The performance of the ensemble RPF method is evaluated in an intersection scenario, and compared to a standard Deep Q-Network method. It is shown that the trained ensemble RPF agent can detect cases with high uncertainty, both in situations that are far from the training distribution, and in situations that seldom occur within the training distribution. In this study, the uncertainty information is used to choose safe actions in unknown situations, which removes all collisions from within the training distribution, and most collisions outside of the distribution.