mlsanityData to 7 Oct 2026

Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach

arXiv 2302.02179 ↗First posted 4 Feb 2023cs.LG cs.AI cs.ROcited by 3

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Driving in dense traffic with human and autonomous drivers is a challenging task that requires high-level planning and reasoning. Human drivers can achieve this task comfortably, and there has been many efforts to model human driver strategies. These strategies can be used as inspirations for developing autonomous driving algorithms or to create high-fidelity simulators. Reinforcement learning is a common tool to model driver policies, but conventional training of these models can be computationally expensive and time-consuming. To address this issue, in this paper, we propose ``skill-based" hierarchical driving strategies, where motion primitives, i.e. skills, are designed and used as high-level actions. This reduces the training time for applications that require multiple models with varying behavior. Simulation results in a merging scenario demonstrate that the proposed approach yields driver models that achieve higher performance with less training compared to baseline reinforcement learning methods.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 22 of its 36 references that are in the map. First posted on arXiv 4 Feb 2023.

Builds on

22 of 36 references.

References · 22

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  3. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,173
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  6. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  7. 2016The Option-Critic Architecturecited by 433
  8. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  9. 2019Dynamics-Aware Unsupervised Discovery of Skillscited by 308
  10. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  11. 2019Efficient Exploration via State Marginal Matchingcited by 186
  12. 2021Reinforcement Learning with Prototypical Representationscited by 179
10 more
  1. 2021Behavior From the Void: Unsupervised Active Pre-Trainingcited by 173
  2. 2019Model-free Deep Reinforcement Learning for Urban Autonomous Drivingcited by 96
  3. 2018Kickstarting Deep Reinforcement Learningcited by 75
  4. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  5. 2018Urban Driving with Multi-Objective Deep Reinforcement Learningcited by 30
  6. 2020Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learningcited by 17
  7. 2021Subgoal-based Reward Shaping to Improve Efficiency in Reinforcement Learningcited by 10
  8. 2020Hierarchical Reinforcement Learning for Self-Driving Decision-Making without Reliance on Labeled Driving Datacited by 7
  9. 2022Designing Rewards for Fast Learningcited by 7
  10. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Hybrid Action Based Reinforcement Learning for Multi-Objective Compatible Autonomous Drivingcited by 7
  2. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Drivingcited by 0
  3. 2025Learning to Drive Safely with Hybrid Optionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Spatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving
  2. 2023Efficient Reinforcement Learning for Autonomous Driving with Parameterized Skills and Priors
  3. 2023Action and Trajectory Planning for Urban Autonomous Driving with Hierarchical Reinforcement Learning
  4. 2022Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills
  5. 2020Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving
  6. 2025Scenario-Based Hierarchical Reinforcement Learning for Automated Driving Decision Making
  7. 2023Interpretable Motion Planner for Urban Driving via Hierarchical Imitation Learning
  8. 2025Goal-conditioned Hierarchical Reinforcement Learning for Sample-efficient and Safe Autonomous Driving at Intersections