# Composing Meta-Policies for Autonomous Driving Using Hierarchical Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1711.01503 · HTML: https://mlsanity.com/paper/1711.01503/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2017.

## Built on by (most cited first, 4 of 4)

- [Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks](https://mlsanity.com/paper/2105.12374/) (2021)
- [Improved Robustness and Safety for Pre-Adaptation of Meta Reinforcement Learning with Prior Regularization](https://mlsanity.com/paper/2108.08448/) (2021)
- [Option Compatible Reward Inverse Reinforcement Learning](https://mlsanity.com/paper/1911.02723/) (2019)
- [Switching Successor Measures for Hierarchical Zero-shot Reinforcement Learning](https://mlsanity.com/paper/2605.13207/) (2026)

## Abstract (arXiv)

Rather than learning new control policies for each new task, it is possible, when tasks share some structure, to compose a "meta-policy" from previously learned policies. This paper reports results from experiments using Deep Reinforcement Learning on a continuous-state, discrete-action autonomous driving simulator. We explore how Deep Neural Networks can represent meta-policies that switch among a set of previously learned policies, specifically in settings where the dynamics of a new scenario are composed of a mixture of previously learned dynamics and where the state observation is possibly corrupted by sensing noise. We also report the results of experiments varying dynamics mixes, distractor policies, magnitudes/distributions of sensing noise, and obstacles. In a fully observed experiment, the meta-policy learning algorithm achieves 2.6x the reward achieved by the next best policy composition technique with 80% less exploration. In a partially observed experiment, the meta-policy learning algorithm converges after 50 iterations while a direct application of RL fails to converge even after 200 iterations.