mlsanityData to 7 Oct 2026

Safer Autonomous Driving in a Stochastic, Partially-Observable Environment by Hierarchical Contingency Planning

arXiv 2204.06509 ↗First posted 13 Apr 2022cs.LG cs.AI cs.ROcited by 0

On the map: End-to-end autonomous driving → Reinforcement learning for driving

Abstract (arXiv)

When learning to act in a stochastic, partially observable environment, an intelligent agent should be prepared to anticipate a change in its belief of the environment state, and be capable of adapting its actions on-the-fly to changing conditions. As humans, we are able to form contingency plans when learning a task with the explicit aim of being able to correct errors in the initial control, and hence prove useful if ever there is a sudden change in our perception of the environment which requires immediate corrective action. This is especially the case for autonomous vehicles (AVs) navigating real-world situations where safety is paramount, and a strong ability to react to a changing belief about the environment is truly needed. In this paper we explore an end-to-end approach, from training to execution, for learning robust contingency plans and combining them with a hierarchical planner to obtain a robust agent policy in an autonomous navigation task where other vehicles' behaviours are unknown, and the agent's belief about these behaviours is subject to sudden, last-second change. We show that our approach results in robust, safe behaviour in a partially observable, stochastic environment, generalizing well over environment dynamics not seen during training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for driving, one of 9 directions in End-to-end autonomous driving. Builds on 20 of its 30 references that are in the map. First posted on arXiv 13 Apr 2022.

Builds on

20 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Addressing Inherent Uncertainty: Risk-Sensitive Behavior Generation for Automated Driving using Distributional Reinforcement Learning5 Feb

Other references · 19

  1. 2015Deep Reinforcement Learning with Double Q-learningcited by 1,704
  2. 2015Prioritized Experience Replaycited by 1,070
  3. 2018Diversity is All You Need: Learning Skills without a Reward Functioncited by 734
  4. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  5. 2019A Survey of Autonomous Driving: Common Practices and Emerging Technologiescited by 373
  6. 2018Safe Exploration in Continuous Action Spacescited by 237
  7. 2016Modular Multitask Reinforcement Learning with Policy Sketchescited by 200
  8. 2020Can Autonomous Vehicles Identify, Recover From, and Adapt to Distribution Shifts?cited by 139
  9. 2017A Laplacian Framework for Option Discovery in Reinforcement Learningcited by 108
  10. 2020One Solution is Not All You Need: Few-Shot Extrapolation via Structured MaxEnt RLcited by 79
  11. 2019Estimating Risk and Uncertainty in Deep Reinforcement Learningcited by 68
  12. 2019Uncertainty Quantification with Statistical Guarantees in End-to-End Autonomous Driving Controlcited by 60
7 more
  1. 2021Hierarchical Reinforcement Learning By Discovering Intrinsic Optionscited by 57
  2. 2019Combining Planning and Deep Reinforcement Learning in Tactical Decision Making for Autonomous Drivingcited by 52
  3. 2018Hierarchical Game-Theoretic Planning for Autonomous Vehiclescited by 48
  4. 2021Contingencies from Observations: Tractable Contingency Planning with Learned Behavior Modelscited by 22
  5. 2019Safe Reinforcement Learning with Scene Decomposition for Navigating Complex Urban Environmentscited by 18
  6. 2020Tactical Decision-Making in Autonomous Driving by Reinforcement Learning with Uncertainty Estimationcited by 14
  7. 2021Learning to Robustly Negotiate Bi-Directional Lane Usage in High-Conflict Driving Scenarioscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Automatically Learning Fallback Strategies with Model-Free Reinforcement Learning in Safety-Critical Driving Scenarios
  2. 2021Building Safer Autonomous Agents by Leveraging Risky Driving Behavior Knowledge
  3. 2022Robust AI Driving Strategy for Autonomous Vehicles
  4. 2025Safe and Nonconservative Contingency Planning for Autonomous Vehicles via Online Learning-Based Reachable Set Barriers
  5. 2022How to Learn from Risk: Explicit Risk-Utility Reinforcement Learning for Efficient and Safe Driving Strategies
  6. 2021Evaluating Robustness over High Level Driving Instruction for Autonomous Driving
  7. 2026Uncertainty-Aware and Temporally Regulated Expert Advice in Reinforcement Learning for Autonomous Driving
  8. 2021Spatially and Seamlessly Hierarchical Reinforcement Learning for State Space and Policy space in Autonomous Driving