# Model-based Lookahead Reinforcement Learning

arXiv: https://arxiv.org/abs/1908.06012 · HTML: https://mlsanity.com/paper/1908.06012/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Aug 2019.

## Built on by (most cited first, 5 of 5)

- [Where to go next: Learning a Subgoal Recommendation Policy for Navigation Among Pedestrians](https://mlsanity.com/paper/2102.13073/) (2021)
- [Look Before Leap: Look-Ahead Planning with Uncertainty in Reinforcement Learning](https://mlsanity.com/paper/2503.20139/) (2025)
- [Machine Learning Meets Advanced Robotic Manipulation](https://mlsanity.com/paper/2309.12560/) (2023)
- [Learning to Plan, Planning to Learn: Adaptive Hierarchical RL-MPC for Sample-Efficient Decision Making](https://mlsanity.com/paper/2512.17091/) (2025)
- [Model-Based Lookahead Reinforcement Learning for in-hand manipulation](https://mlsanity.com/paper/2510.08884/) (2025)

## Abstract (arXiv)

Model-based Reinforcement Learning (MBRL) allows data-efficient learning which is required in real world applications such as robotics. However, despite the impressive data-efficiency, MBRL does not achieve the final performance of state-of-the-art Model-free Reinforcement Learning (MFRL) methods. We leverage the strengths of both realms and propose an approach that obtains high performance with a small amount of data. In particular, we combine MFRL and Model Predictive Control (MPC). While MFRL's strength in exploration allows us to train a better forward dynamics model for MPC, MPC improves the performance of the MFRL policy by sampling-based planning. The experimental results in standard continuous control benchmarks show that our approach can achieve MFRL`s level of performance while being as data-efficient as MBRL.