# Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Data

arXiv: https://arxiv.org/abs/2001.09816 · HTML: https://mlsanity.com/paper/2001.09816/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Jan 2020.

## Built on by (most cited first, 7 of 7)

- [Runtime Safety Assurance for Learning-enabled Control of Autonomous Driving Vehicles](https://mlsanity.com/paper/2109.13446/) (2021)
- [Integrated Decision and Control at Multi-Lane Intersections with Mixed Traffic Flow](https://mlsanity.com/paper/2108.13038/) (2021)
- [Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach](https://mlsanity.com/paper/2302.02179/) (2023)
- [From Multi-agent to Multi-robot: A Scalable Training and Evaluation Platform for Multi-robot Reinforcement Learning](https://mlsanity.com/paper/2206.09590/) (2022)
- [Decision-Making under On-Ramp merge Scenarios by Distributional Soft Actor-Critic Algorithm](https://mlsanity.com/paper/2103.04535/) (2021)
- [Integrated Decision and Control for High-Level Automated Vehicles by Mixed Policy Gradient and Its Experiment Verification](https://mlsanity.com/paper/2210.10613/) (2022)
- [Distributional Soft Actor-Critic with Harmonic Gradient for Safe and Efficient Autonomous Driving in Multi-lane Scenarios](https://mlsanity.com/paper/2505.13532/) (2025)

## Abstract (arXiv)

Decision making for self-driving cars is usually tackled by manually encoding rules from drivers' behaviors or imitating drivers' manipulation using supervised learning techniques. Both of them rely on mass driving data to cover all possible driving scenarios. This paper presents a hierarchical reinforcement learning method for decision making of self-driving cars, which does not depend on a large amount of labeled driving data. This method comprehensively considers both high-level maneuver selection and low-level motion control in both lateral and longitudinal directions. We firstly decompose the driving tasks into three maneuvers, including driving in lane, right lane change and left lane change, and learn the sub-policy for each maneuver. Then, a master policy is learned to choose the maneuver policy to be executed in the current state. All policies including master policy and maneuver policies are represented by fully-connected neural networks and trained by using asynchronous parallel reinforcement learners (APRL), which builds a mapping from the sensory outputs to driving decisions. Different state spaces and reward functions are designed for each maneuver. We apply this method to a highway driving scenario, which demonstrates that it can realize smooth and safe decision making for self-driving cars.