# Learning hierarchical behavior and motion planning for autonomous driving

arXiv: https://arxiv.org/abs/2005.03863 · HTML: https://mlsanity.com/paper/2005.03863/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2020.

## Built on by (most cited first, 15 of 15)

- [NEAT: Neural Attention Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2109.04456/) (2021)
- [Reward (Mis)design for Autonomous Driving](https://mlsanity.com/paper/2104.13906/) (2021)
- [ADAPT: Action-aware Driving Caption Transformer](https://mlsanity.com/paper/2302.00673/) (2023)
- [Augmenting Reinforcement Learning with Transformer-based Scene Representation Learning for Decision-making of Autonomous Driving](https://mlsanity.com/paper/2208.12263/) (2022)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [Imitation Learning of Hierarchical Driving Model: from Continuous Intention to Continuous Trajectory](https://mlsanity.com/paper/2010.10393/) (2020)
- [EnsembleFollower: A Hybrid Car-Following Framework Based On Reinforcement Learning and Hierarchical Planning](https://mlsanity.com/paper/2308.16008/) (2023)
- [Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning](https://mlsanity.com/paper/2303.13986/) (2023)
- [CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Driving](https://mlsanity.com/paper/2505.21581/) (2025)
- [Extensive Exploration in Complex Traffic Scenarios using Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2501.14992/) (2025)
- [AGPNet -- Autonomous Grading Policy Network](https://mlsanity.com/paper/2112.10877/) (2021)
- [A Survey on Hybrid Motion Planning Methods for Automated Driving Systems](https://mlsanity.com/paper/2406.05575/) (2024)
- [Zero-shot Transfer Learning of Driving Policy via Socially Adversarial Traffic Flow](https://mlsanity.com/paper/2304.12821/) (2023)
- [Attention-Driven Multi-Agent Reinforcement Learning: Enhancing Decisions with Expertise-Informed Tasks](https://mlsanity.com/paper/2404.05840/) (2024)
- [Bootstrapping Reinforcement Learning with Sub-optimal Policies for Autonomous Driving](https://mlsanity.com/paper/2509.04712/) (2025)

## Abstract (arXiv)

Learning-based driving solution, a new branch for autonomous driving, is expected to simplify the modeling of driving by learning the underlying mechanisms from data. To improve the tactical decision-making for learning-based driving solution, we introduce hierarchical behavior and motion planning (HBMP) to explicitly model the behavior in learning-based solution. Due to the coupled action space of behavior and motion, it is challenging to solve HBMP problem using reinforcement learning (RL) for long-horizon driving tasks. We transform HBMP problem by integrating a classical sampling-based motion planner, of which the optimal cost is regarded as the rewards for high-level behavior learning. As a result, this formulation reduces action space and diversifies the rewards without losing the optimality of HBMP. In addition, we propose a sharable representation for input sensory data across simulation platforms and real-world environment, so that models trained in a fast event-based simulator, SUMO, can be used to initialize and accelerate the RL training in a dynamics based simulator, CARLA. Experimental results demonstrate the effectiveness of the method. Besides, the model is successfully transferred to the real-world, validating the generalization capability.