mlsanityData to 7 Oct 2026

A Risk-Averse Preview-based Q-Learning Algorithm: Application to Highway Driving of Autonomous Vehicles

arXiv 2112.03232 ↗First posted 6 Dec 2021eess.SY cs.SYcited by 1

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

A risk-averse preview-based Q-learning planner is presented for navigation of autonomous vehicles. To this end, the multi-lane road ahead of a vehicle is represented by a finite-state non-stationary Markov decision process (MDP). A risk assessment unit module is then presented that leverages the preview information provided by sensors along with a stochastic reachability module to assign reward values to the MDP states and update them as scenarios develop. A sampling-based risk-averse preview-based Q-learning algorithm is finally developed that generates samples using the preview information and reward function to learn risk-averse optimal planning strategies without actual interaction with the environment. The risk factor is imposed on the objective function to avoid fluctuation of the Q values, which can jeopardize the vehicle's safety and/or performance. The overall hybrid automaton model of the system is leveraged to develop a feasibility check unit module that detects unfeasible plans and enables the planner system to react proactively to the changes of the environment. Finally, to verify the efficiency of the presented algorithm, its implementation on two highway driving scenarios of an autonomous vehicle in a varying traffic density is considered.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 26 of its 93 references that are in the map. First posted on arXiv 6 Dec 2021.

Builds on

26 of 93 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021A Survey of Deep Reinforcement Learning Algorithms for Motion Planning and Control of Autonomous Vehicles29 May

Other references · 25

  1. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 320
  2. 2013Risk-sensitive Reinforcement Learningcited by 129
  3. 2014Optimizing the CVaR via Samplingcited by 98
  4. 2017Probabilistic Vehicle Trajectory Prediction over Occupancy Grid Map via Recurrent Neural Networkcited by 56
  5. 2014Policy Gradients for CVaR-Constrained MDPscited by 20
  6. 2019An Efficient Reachability-Based Framework for Provably Safe Autonomous Navigation in Unknown Environmentscited by 18
  7. 2019A Risk-Sensitive Finite-Time Reachability Approach for Safety of Stochastic Dynamic Systemscited by 14
  8. 2019Risk-Averse Planning Under Uncertaintycited by 11
  9. 2020Driver Modeling through Deep Reinforcement Learning and Behavioral Game Theorycited by 8
  10. 2021Safe Value Functionscited by 7
  11. 2021A Taxonomy of Similarity Metrics for Markov Decision Processescited by 6
  12. 2020Data-driven Dynamic Multi-objective Optimal Control: An Aspiration-satisfying Reinforcement Learning Approachcited by 5
13 more
  1. 2015A Framework for Time-Consistent, Risk-Averse Model Predictive Control: Theory and Algorithmscited by 4
  2. 2017Risk-averse model predictive controlcited by 4
  3. 2016Performance guarantees for model-based Approximate Dynamic Programming in continuous spacescited by 3
  4. 2021Risk-Averse RRT* Planning with Nonlinear Steering and Tracking Controllers for Nonlinear Robotic Systems Under Uncertaintycited by 3
  5. 2021Assured Learning-enabled Autonomy: A Metacognitive Reinforcement Learning Frameworkcited by 3
  6. 2020Data-Driven Hierarchical Predictive Learning in Unknown Environmentscited by 2
  7. 2020Online Learning Based Risk-Averse Stochastic MPC of Constrained Linear Uncertain Systemscited by 2
  8. 2021A Convex Programming Approach to Data-Driven Risk-Averse Reinforcement Learningcited by 2
  9. 2015Stochastic Optimal Growth Model with Risk Sensitive Preferencescited by 1
  10. 2019Safety Control with Preview Automatoncited by 1
  11. 2019Risk-sensitive safety specifications for stochastic systems using Conditional Value-at-Riskcited by 1
  12. 2019Long-run risk sensitive impulse controlcited by 1
  13. 2020Learning-Based Risk-Averse Model Predictive Control for Adaptive Cruise Control with Stochastic Driver Modelscited by 1

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Risk-Averse Total-Reward Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhanced Safety in Autonomous Driving: Integrating Latent State Diffusion Model for End-to-End Navigation
  2. 2021Risk Averse Bayesian Reward Learning for Autonomous Navigation from Human Demonstration
  3. 2022How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
  4. 2021How To Not Drive: Learning Driving Constraints from Demonstration
  5. 2021Don't Get Yourself into Trouble! Risk-aware Decision-Making for Autonomous Vehicles
  6. 2025Risk-Aware Reinforcement Learning for Autonomous Driving: Improving Safety When Driving through Intersection
  7. 2023Risk-Aware Reward Shaping of Reinforcement Learning Agents for Autonomous Driving
  8. 2026Bridging Predictive Uncertainty and Safe Action: Sample-Conditioned Differentiable Planning for Autonomous Driving