# Data-Efficient Reinforcement Learning with Probabilistic Model Predictive Control

arXiv: https://arxiv.org/abs/1706.06491 · HTML: https://mlsanity.com/paper/1706.06491/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2017.

## Built on by (most cited first, 30 of 53)

- [Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning](https://mlsanity.com/paper/2108.06266/) (2021)
- [A Review of Safe Reinforcement Learning: Methods, Theory and Applications](https://mlsanity.com/paper/2205.10330/) (2022)
- [Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentation](https://mlsanity.com/paper/2202.06558/) (2022)
- [Continuous-Time Model-Based Reinforcement Learning](https://mlsanity.com/paper/2102.04764/) (2021)
- [Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rollouts](https://mlsanity.com/paper/2105.03363/) (2021)
- [Vision-Based Autonomous Car Racing Using Deep Imitative Reinforcement Learning](https://mlsanity.com/paper/2107.08325/) (2021)
- [Is Model Ensemble Necessary? Model-based RL via a Single Model with Lipschitz Regularized Value Function](https://mlsanity.com/paper/2302.01244/) (2023)
- [Model-based Meta Reinforcement Learning using Graph Structured Surrogate Models](https://mlsanity.com/paper/2102.08291/) (2021)
- [An Experimental Design Perspective on Model-Based Reinforcement Learning](https://mlsanity.com/paper/2112.05244/) (2021)
- [Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learning](https://mlsanity.com/paper/2103.10369/) (2021)
- [DMWM: Dual-Mind World Model with Long-Term Imagination](https://mlsanity.com/paper/2502.07591/) (2025)
- [Exploration via Planning for Information about the Optimal Trajectory](https://mlsanity.com/paper/2210.04642/) (2022)
- [Function-space Parameterization of Neural Networks for Sequential Learning](https://mlsanity.com/paper/2403.10929/) (2024)
- [Gradient-Based Trajectory Optimization With Learned Dynamics](https://mlsanity.com/paper/2204.04558/) (2022)
- [Posterior Coreset Construction with Kernelized Stein Discrepancy for Model-Based Reinforcement Learning](https://mlsanity.com/paper/2206.01162/) (2022)
- [Event-Triggered Model Predictive Control with Deep Reinforcement Learning for Autonomous Driving](https://mlsanity.com/paper/2208.10302/) (2022)
- [Effects of Safety State Augmentation on Safe Exploration](https://mlsanity.com/paper/2206.02675/) (2022)
- [Comparison of Model Predictive Control and Proximal Policy Optimization for a 1-DOF Helicopter System](https://mlsanity.com/paper/2408.15633/) (2024)
- [The Crucial Role of Problem Formulation in Real-World Reinforcement Learning](https://mlsanity.com/paper/2503.20442/) (2025)
- [Bridging Model-based Safety and Model-free Reinforcement Learning through System Identification of Low Dimensional Linear Models](https://mlsanity.com/paper/2205.05787/) (2022)
- [Computationally and Sample Efficient Safe Reinforcement Learning Using Adaptive Conformal Prediction](https://mlsanity.com/paper/2503.17678/) (2025)
- [Optimal kernel regression bounds under energy-bounded noise](https://mlsanity.com/paper/2505.22235/) (2025)
- [Mind the Uncertainty: Risk-Aware and Actively Exploring Model-Based Reinforcement Learning](https://mlsanity.com/paper/2309.05582/) (2023)
- [Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling](https://mlsanity.com/paper/2410.04988/) (2024)
- [Active Alignments of Lens Systems with Reinforcement Learning](https://mlsanity.com/paper/2503.02075/) (2025)
- [Optimal Control of Probabilistic Dynamics Models via Mean Hamiltonian Minimization](https://mlsanity.com/paper/2504.02543/) (2025)
- [Spectral Representation-based Reinforcement Learning](https://mlsanity.com/paper/2512.15036/) (2025)
- [Bellman: A Toolbox for Model-Based Reinforcement Learning in TensorFlow](https://mlsanity.com/paper/2103.14407/) (2021)
- [Data-Efficient Reinforcement Learning for Malaria Control](https://mlsanity.com/paper/2105.01620/) (2021)
- [Proximal Policy Optimization for Tracking Control Exploiting Future Reference Information](https://mlsanity.com/paper/2107.09647/) (2021)

## Abstract (arXiv)

Trial-and-error based reinforcement learning (RL) has seen rapid advancements in recent times, especially with the advent of deep neural networks. However, the majority of autonomous RL algorithms require a large number of interactions with the environment. A large number of interactions may be impractical in many real-world applications, such as robotics, and many practical systems have to obey limitations in the form of state space or control constraints. To reduce the number of system interactions while simultaneously handling constraints, we propose a model-based RL framework based on probabilistic Model Predictive Control (MPC). In particular, we propose to learn a probabilistic transition model using Gaussian Processes (GPs) to incorporate model uncertainty into long-term predictions, thereby, reducing the impact of model errors. We then use MPC to find a control sequence that minimises the expected long-term cost. We provide theoretical guarantees for first-order optimality in the GP-based transition models with deterministic approximate inference for long-term planning. We demonstrate that our approach does not only achieve state-of-the-art data efficiency, but also is a principled way for RL in constrained environments.