mlsanityData to 7 Oct 2026

Risk-anticipatory autonomous driving strategies considering vehicles' weights, based on hierarchical deep reinforcement learning

arXiv 2401.08661 ↗First posted 27 Dec 2023cs.RO cs.LGcited by 3

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

Autonomous vehicles (AVs) have the potential to prevent accidents caused by drivers errors and reduce road traffic risks. Due to the nature of heavy vehicles, whose collisions cause more serious crashes, the weights of vehicles need to be considered when making driving strategies aimed at reducing the potential risks and their consequences in the context of autonomous driving. This study develops an autonomous driving strategy based on risk anticipation, considering the weights of surrounding vehicles and using hierarchical deep reinforcement learning. A risk indicator integrating surrounding vehicles weights, based on the risk field theory, is proposed and incorporated into autonomous driving decisions. A hybrid action space is designed to allow for left lane changes, right lane changes and car-following, which enables AVs to act more freely and realistically whenever possible. To solve the above hybrid decision-making problem, a hierarchical proximal policy optimization (HPPO) algorithm with an attention mechanism (AT-HPPO) is developed, providing great advantages in maintaining stable performance with high robustness and generalization. An indicator, potential collision energy in conflicts (PCEC), is newly proposed to evaluate the performance of the developed AV driving strategy from the perspective of the consequences of potential accidents. The performance evaluation results in simulation and dataset demonstrate that our model provides driving strategies that reduce both the likelihood and consequences of potential accidents, at the same time maintaining driving efficiency. The developed method is especially meaningful for AVs driving on highways, where heavy vehicles make up a high proportion of the traffic.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 21 of its 98 references that are in the map. First posted on arXiv 27 Dec 2023.

Builds on

21 of 98 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Don't Get Yourself into Trouble! Risk-aware Decision-Making for Autonomous Vehicles8 Jun

Other references · 20

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  6. 2019A Survey of Deep Learning Techniques for Autonomous Drivingcited by 401
  7. 2018The highD Dataset: A Drone Dataset of Naturalistic Vehicle Trajectories on German Highways for Validation of Highly Automated Driving Systemscited by 276
  8. 2023Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Surveycited by 101
  9. 2020A causal view of compositional zero-shot recognitioncited by 78
  10. 2023Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learningcited by 72
  11. 2019Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Spacecited by 53
  12. 2019Human-Like Autonomous Car-Following Model with Deep Reinforcement Learningcited by 47
8 more
  1. 2018A Reinforcement Learning Based Approach for Automated Lane Change Maneuverscited by 44
  2. 2020Automated Lane Change Strategy using Proximal Policy Optimization-based Deep Reinforcement Learningcited by 30
  3. 2023Data-driven Traffic Simulation: A Comprehensive Reviewcited by 29
  4. 2019Autonomous Highway Driving using Deep Reinforcement Learningcited by 22
  5. 2019Comparison of Deep Reinforcement Learning and Model Predictive Control for Adaptive Cruise Controlcited by 19
  6. 2020A Learning-based Discretionary Lane-Change Decision-Making Model with Driving Style Awarenesscited by 9
  7. 2020Decision-making Strategy on Highway for Autonomous Vehicles using Deep Reinforcement Learningcited by 6
  8. 2022Predicting highway lane-changing maneuvers: A benchmark analysis of machine and ensemble learning algorithmscited by 3

Built on by

Cited by year

Citing papers per year: 2025: 3.

3 papers.

  1. 2025Towards a Reward-Free Reinforcement Learning Framework for Vehicle Controlcited by 0
  2. 2025Multi-Timescale Hierarchical Reinforcement Learning for Unified Behavior and Control of Autonomous Drivingcited by 0
  3. 2025Orchestrate, Generate, Reflect: A VLM-Based Multi-Agent Collaboration Framework for Automated Driving Policy Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection
  2. 2021Reinforcement Learning Based Safe Decision Making for Highway Autonomous Driving
  3. 2022Prediction Based Decision Making for Autonomous Highway Driving
  4. 2025Goal-conditioned Hierarchical Reinforcement Learning for Sample-efficient and Safe Autonomous Driving at Intersections
  5. 2026Autonomous Vehicle Collision Avoidance With Racing Parameterized Deep Reinforcement Learning
  6. 2024Deep Reinforcement Learning for Advanced Longitudinal Control and Collision Avoidance in High-Risk Driving Scenarios
  7. 2021A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning
  8. 2024Research on Autonomous Driving Decision-making Strategies based Deep Reinforcement Learning