# Learning to Drive Safely with Hybrid Options

arXiv: https://arxiv.org/abs/2510.24674 · HTML: https://mlsanity.com/paper/2510.24674/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for autonomous driving and traffic, one of 15 directions in Deep reinforcement learning methods.
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 28 Oct 2025.

## Placed in

- Deep reinforcement learning methods / [RL for autonomous driving and traffic](https://mlsanity.com/deeprl/rl-for-autonomous-driving-and-traffic/)

## Builds on (most cited first, 24 of 24)

- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Deep Reinforcement Learning with Double Q-learning](https://mlsanity.com/paper/1509.06461/) (2015)
- [Constrained Policy Optimization](https://mlsanity.com/paper/1705.10528/) (2017)
- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [The Option-Critic Architecture](https://mlsanity.com/paper/1609.05140/) (2016)
- [Accelerating Reinforcement Learning with Learned Skill Priors](https://mlsanity.com/paper/2010.11944/) (2020)
- [Survey of Deep Reinforcement Learning for Motion Planning of Autonomous Vehicles](https://mlsanity.com/paper/2001.11231/) (2020)
- [MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies](https://mlsanity.com/paper/1905.09808/) (2019)
- [A Deep Hierarchical Approach to Lifelong Learning in Minecraft](https://mlsanity.com/paper/1604.07255/) (2016)
- [The Option Keyboard: Combining Skills in Reinforcement Learning](https://mlsanity.com/paper/2106.13105/) (2021)
- [Reinforcement Learning with Parameterized Actions](https://mlsanity.com/paper/1509.01644/) (2015)
- [Parametrized Deep Q-Networks Learning: Reinforcement Learning with Discrete-Continuous Hybrid Action Space](https://mlsanity.com/paper/1810.06394/) (2018)
- [Hybrid Actor-Critic Reinforcement Learning in Parameterized Action Space](https://mlsanity.com/paper/1903.01344/) (2019)
- [Continuous-Discrete Reinforcement Learning for Hybrid Control in Robotics](https://mlsanity.com/paper/2001.00449/) (2020)
- [DAC: The Double Actor-Critic Architecture for Learning Options](https://mlsanity.com/paper/1904.12691/) (2019)
- [Driving in Dense Traffic with Model-Free Reinforcement Learning](https://mlsanity.com/paper/1909.06710/) (2019)
- [Autonomous Highway Driving using Deep Reinforcement Learning](https://mlsanity.com/paper/1904.00035/) (2019)
- [SKID RAW: Skill Discovery from Raw Trajectories](https://mlsanity.com/paper/2103.14610/) (2021)
- [Automated Lane Change Decision Making using Deep Reinforcement Learning in Dynamic and Uncertain Highway Environment](https://mlsanity.com/paper/1909.11538/) (2019)
- [Trajectory Planning for Autonomous Vehicles Using Hierarchical Reinforcement Learning](https://mlsanity.com/paper/2011.04752/) (2020)
- [Boosting Offline Reinforcement Learning for Autonomous Driving with Hierarchical Latent Skills](https://mlsanity.com/paper/2309.13614/) (2023)
- [Accelerating Reinforcement Learning for Autonomous Driving using Task-Agnostic and Ego-Centric Motion Skills](https://mlsanity.com/paper/2209.12072/) (2022)
- [Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approach](https://mlsanity.com/paper/2302.02179/) (2023)
- [A Dual Perspective of Reinforcement Learning for Imposing Policy Constraints](https://mlsanity.com/paper/2404.16468/) (2024)

## Abstract (arXiv)

Out of the many deep reinforcement learning approaches for autonomous driving, only few make use of the options (or skills) framework. That is surprising, as this framework is naturally suited for hierarchical control applications in general, and autonomous driving tasks in specific. Therefore, in this work the options framework is applied and tailored to autonomous driving tasks on highways. More specifically, we define dedicated options for longitudinal and lateral manoeuvres with embedded safety and comfort constraints. This way, prior domain knowledge can be incorporated into the learning process and the learned driving behaviour can be constrained more easily. We propose several setups for hierarchical control with options and derive practical algorithms following state-of-the-art reinforcement learning techniques. By separately selecting actions for longitudinal and lateral control, the introduced policies over combined and hybrid options obtain the same expressiveness and flexibility that human drivers have, while being easier to interpret than classical policies over continuous actions. Of all the investigated approaches, these flexible policies over hybrid options perform the best under varying traffic conditions, outperforming the baseline policies over actions.