# Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Driving

arXiv: https://arxiv.org/abs/1905.02680 · HTML: https://mlsanity.com/paper/1905.02680/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 May 2019.

## Built on by (most cited first, 30 of 52)

- [BEVerse: Unified Perception and Prediction in Birds-Eye-View for Vision-Centric Autonomous Driving](https://mlsanity.com/paper/2205.09743/) (2022)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [A Review of Reward Functions for Reinforcement Learning in the context of Autonomous Driving](https://mlsanity.com/paper/2405.01440/) (2024)
- [Ensemble Quantile Networks: Uncertainty-Aware Reinforcement Learning with Applications in Autonomous Driving](https://mlsanity.com/paper/2105.10266/) (2021)
- [Beyond Games: A Systematic Review of Neural Monte Carlo Tree Search Applications](https://mlsanity.com/paper/2303.08060/) (2023)
- [Robustness and Adaptability of Reinforcement Learning based Cooperative Autonomous Driving in Mixed-autonomy Traffic](https://mlsanity.com/paper/2202.00881/) (2022)
- [BetaZero: Belief-State Planning for Long-Horizon POMDPs using Learned Approximations](https://mlsanity.com/paper/2306.00249/) (2023)
- [HighwayLLM: Decision-Making and Navigation in Highway Driving with RL-Informed Language Model](https://mlsanity.com/paper/2405.13547/) (2024)
- [Car-Following Models: A Multidisciplinary Review](https://mlsanity.com/paper/2304.07143/) (2023)
- [Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2306.15968/) (2023)
- [C-MCTS: Safe Planning with Monte Carlo Tree Search](https://mlsanity.com/paper/2305.16209/) (2023)
- [Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving](https://mlsanity.com/paper/2105.06517/) (2021)
- [iPLAN: Intent-Aware Planning in Heterogeneous Traffic via Distributed Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2306.06236/) (2023)
- [Hierarchical End-to-End Autonomous Driving: Integrating BEV Perception with Deep Reinforcement Learning](https://mlsanity.com/paper/2409.17659/) (2024)
- [Using Common Random Numbers for Simulation-based Planning with Rollouts](https://mlsanity.com/paper/2605.04732/) (2026)
- [Reinforcement Learning based Negotiation-aware Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2107.03600/) (2021)
- [How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies](https://mlsanity.com/paper/2203.08409/) (2022)
- [Optimizing Trajectories for Highway Driving with Offline Reinforcement Learning](https://mlsanity.com/paper/2203.10949/) (2022)
- [Prediction Based Decision Making for Autonomous Highway Driving](https://mlsanity.com/paper/2209.02106/) (2022)
- [Dynamic Simplex: Balancing Safety and Performance in Autonomous Cyber Physical Systems](https://mlsanity.com/paper/2302.09750/) (2023)
- [Reinforcement Learning for Industrial Control Network Cyber Security Orchestration](https://mlsanity.com/paper/2106.05332/) (2021)
- [GIN: Graph-based Interaction-aware Constraint Policy Optimization for Autonomous Driving](https://mlsanity.com/paper/2206.01488/) (2022)
- [A Survey on Explainable Reinforcement Learning: Concepts, Algorithms, Challenges](https://mlsanity.com/paper/2211.06665/) (2022)
- [Decentralized Cooperative Lane Changing at Freeway Weaving Areas Using Multi-Agent Deep Reinforcement Learning](https://mlsanity.com/paper/2110.08124/) (2021)
- [Off Environment Evaluation Using Convex Risk Minimization](https://mlsanity.com/paper/2112.11532/) (2021)
- [Bi-Level Optimization Augmented with Conditional Variational Autoencoder for Autonomous Driving in Dense Traffic](https://mlsanity.com/paper/2212.02224/) (2022)
- [n-Step Temporal Difference Learning with Optimal n](https://mlsanity.com/paper/2303.07068/) (2023)
- [End-to-End Learning of Behavioural Inputs for Autonomous Driving in Dense Traffic](https://mlsanity.com/paper/2310.14766/) (2023)
- [Hi-Drive: Hierarchical POMDP Planning for Safe Autonomous Driving in Diverse Urban Environments](https://mlsanity.com/paper/2409.18411/) (2024)
- [BIDA: A Bi-level Interaction Decision-making Algorithm for Autonomous Vehicles in Dynamic Traffic Scenarios](https://mlsanity.com/paper/2506.16546/) (2025)

## Abstract (arXiv)

Tactical decision making for autonomous driving is challenging due to the diversity of environments, the uncertainty in the sensor information, and the complex interaction with other road users. This paper introduces a general framework for tactical decision making, which combines the concepts of planning and learning, in the form of Monte Carlo tree search and deep reinforcement learning. The method is based on the AlphaGo Zero algorithm, which is extended to a domain with a continuous state space where self-play cannot be used. The framework is applied to two different highway driving cases in a simulated environment and it is shown to perform better than a commonly used baseline method. The strength of combining planning and learning is also illustrated by a comparison to using the Monte Carlo tree search or the neural network policy separately.