# Reinforcement Learning based Control of Imitative Policies for Near-Accident Driving

arXiv: https://arxiv.org/abs/2007.00178 · HTML: https://mlsanity.com/paper/2007.00178/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Imitation and offline RL for driving, one of 13 directions in Offline RL and imitation learning from data.
- First posted on arXiv 1 Jul 2020.

## Placed in

- Offline RL and imitation learning from data / [Imitation and offline RL for driving](https://mlsanity.com/offlinerl/imitation-and-offline-rl-for-driving/)

## Built on by (most cited first, 23 of 23)

- [Semantics for Robotic Mapping, Perception and Interaction: A Survey](https://mlsanity.com/paper/2101.00443/) (2021)
- [Efficient Deep Reinforcement Learning with Imitative Expert Priors for Autonomous Driving](https://mlsanity.com/paper/2103.10690/) (2021)
- [Learning Multimodal Rewards from Rankings](https://mlsanity.com/paper/2109.12750/) (2021)
- [Learning from Imperfect Demonstrations from Agents with Varying Dynamics](https://mlsanity.com/paper/2103.05910/) (2021)
- [A Safe Hierarchical Planning Framework for Complex Driving Scenarios based on Reinforcement Learning](https://mlsanity.com/paper/2101.06778/) (2021)
- [On the convergence of policy gradient methods to Nash equilibria in general stochastic games](https://mlsanity.com/paper/2210.08857/) (2022)
- [DQ-GAT: Towards Safe and Efficient Autonomous Driving with Deep Q-Learning and Graph Attention Networks](https://mlsanity.com/paper/2108.05030/) (2021)
- [Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills](https://mlsanity.com/paper/2209.12072/) (2022)
- [Mixed Traffic Control and Coordination from Pixels](https://mlsanity.com/paper/2302.09167/) (2023)
- [Leveraging Smooth Attention Prior for Multi-Agent Trajectory Prediction](https://mlsanity.com/paper/2203.04421/) (2022)
- [Neural Amortized Inference for Nested Multi-agent Reasoning](https://mlsanity.com/paper/2308.11071/) (2023)
- [Receding Horizon Planning with Rule Hierarchies for Autonomous Vehicles](https://mlsanity.com/paper/2212.03323/) (2022)
- [Personalized Federated Learning of Driver Prediction Models for Autonomous Driving](https://mlsanity.com/paper/2112.00956/) (2021)
- [Learning Sequential Acquisition Policies for Robot-Assisted Feeding](https://mlsanity.com/paper/2309.05197/) (2023)
- [Human-AI Shared Control via Policy Dissection](https://mlsanity.com/paper/2206.00152/) (2022)
- [Compensating for Sensing Failures via Delegation in Human-AI Hybrid Systems](https://mlsanity.com/paper/2303.01300/) (2023)
- [DynaMITE-RL: A Dynamic Model for Improved Temporal Meta-Reinforcement Learning](https://mlsanity.com/paper/2402.15957/) (2024)
- [Optimizing Delegation in Collaborative Human-AI Hybrid Teams](https://mlsanity.com/paper/2402.05605/) (2024)
- [Fail-Safe Adversarial Generative Imitation Learning](https://mlsanity.com/paper/2203.01696/) (2022)
- [Risk-aware Meta-level Decision Making for Exploration Under Uncertainty](https://mlsanity.com/paper/2209.05580/) (2022)
- [Act Better by Timing: A timing-Aware Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2406.13223/) (2024)
- [Multi-Agent Inverse Q-Learning from Demonstrations](https://mlsanity.com/paper/2503.04679/) (2025)
- [CommonRoad-Game: A Human-in-the-Loop Simulation Framework for Autonomous Driving](https://mlsanity.com/paper/2607.01382/) (2026)

## Abstract (arXiv)

Autonomous driving has achieved significant progress in recent years, but autonomous cars are still unable to tackle high-risk situations where a potential accident is likely. In such near-accident scenarios, even a minor change in the vehicle's actions may result in drastically different consequences. To avoid unsafe actions in near-accident scenarios, we need to fully explore the environment. However, reinforcement learning (RL) and imitation learning (IL), two widely-used policy learning methods, cannot model rapid phase transitions and are not scalable to fully cover all the states. To address driving in near-accident scenarios, we propose a hierarchical reinforcement and imitation learning (H-ReIL) approach that consists of low-level policies learned by IL for discrete driving modes, and a high-level policy learned by RL that switches between different driving modes. Our approach exploits the advantages of both IL and RL by integrating them into a unified learning framework. Experimental results and user studies suggest our approach can achieve higher efficiency and safety compared to other methods. Analyses of the policies demonstrate our high-level policy appropriately switches between different low-level policies in near-accident driving situations.