# A Hierarchical Architecture for Sequential Decision-Making in Autonomous Driving using Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1906.08464 · HTML: https://mlsanity.com/paper/1906.08464/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2019.

## Built on by (most cited first, 4 of 4)

- [Safe Navigation: Training Autonomous Vehicles using Deep Reinforcement Learning in CARLA](https://mlsanity.com/paper/2311.10735/) (2023)
- [Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2501.14992/) (2025)
- [LaksNet: an end-to-end deep learning model for self-driving cars in Udacity simulator](https://mlsanity.com/paper/2310.16103/) (2023)
- [Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving](https://mlsanity.com/paper/2509.04712/) (2025)

## Abstract (arXiv)

Tactical decision making is a critical feature for advanced driving systems, that incorporates several challenges such as complexity of the uncertain environment and reliability of the autonomous system. In this work, we develop a multi-modal architecture that includes the environmental modeling of ego surrounding and train a deep reinforcement learning (DRL) agent that yields consistent performance in stochastic highway driving scenarios. To this end, we feed the occupancy grid of the ego surrounding into the DRL agent and obtain the high-level sequential commands (i.e. lane change) to send them to lower-level controllers. We will show that dividing the autonomous driving problem into a multi-layer control architecture enables us to leverage the AI power to solve each layer separately and achieve an admissible reliability score. Comparing with end-to-end approaches, this architecture enables us to end up with a more reliable system which can be implemented in actual self-driving cars.