# Curious Meta-Controller: Adaptive Alternation between Model-Based and Model-Free Control in Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1905.01718 · HTML: https://mlsanity.com/paper/1905.01718/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2019.

## Built on by (most cited first, 3 of 3)

- [Temporal Difference Learning for Model Predictive Control](https://mlsanity.com/paper/2203.04955/) (2022)
- [Improving Model-Based Reinforcement Learning with Internal State Representations through Self-Supervision](https://mlsanity.com/paper/2102.05599/) (2021)
- [Model Predictive Control with Self-supervised Representation Learning](https://mlsanity.com/paper/2304.07219/) (2023)

## Abstract (arXiv)

Recent success in deep reinforcement learning for continuous control has been dominated by model-free approaches which, unlike model-based approaches, do not suffer from representational limitations in making assumptions about the world dynamics and model errors inevitable in complex domains. However, they require a lot of experiences compared to model-based approaches that are typically more sample-efficient. We propose to combine the benefits of the two approaches by presenting an integrated approach called Curious Meta-Controller. Our approach alternates adaptively between model-based and model-free control using a curiosity feedback based on the learning progress of a neural model of the dynamics in a learned latent space. We demonstrate that our approach can significantly improve the sample efficiency and achieve near-optimal performance on learning robotic reaching and grasping tasks from raw-pixel input in both dense and sparse reward settings.