Data to 7 Oct 2026

Model-Based Reinforcement Learning via Meta-Policy Optimization

arXiv 1809.05214 ↗First posted 14 Sep 2018cs.LG cs.AI stat.MLcited by 99

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning approaches carry the promise of being data efficient. However, due to challenges in learning dynamics models that sufficiently match the real-world dynamics, they struggle to achieve the same asymptotic performance as model-free methods. We propose Model-Based Meta-Policy-Optimization (MB-MPO), an approach that foregoes the strong reliance on accurate learned dynamics models. Using an ensemble of learned dynamic models, MB-MPO meta-learns a policy that can quickly adapt to any model in the ensemble with one policy gradient step. This steers the meta-policy towards internalizing consistent dynamics predictions among the ensemble while shifting the burden of behaving optimally w.r.t. the model discrepancies towards the adaptation step. Our experiments show that MB-MPO is more robust to model imperfections than previous model-based approaches. Finally, we demonstrate that our approach is able to match the asymptotic performance of model-free methods while requiring significantly less experience.

Key facts. Cited by 99 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2020: 2; 2021: 26; 2022: 18; 2023: 15; 2024: 19; 2025: 11; 2026: 8.

99 papers.

Sort:

By area:

2020MOPO: Model-based Offline Policy Optimizationcited by 630
2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
2021Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyondcited by 151
2022Neural-Fly Enables Rapid Learning for Agile Flight in Strong Windscited by 80
2024Keypoint Action Tokens Enable In-Context Imitation Learning in Roboticscited by 71
2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 59
2022Investigating Compounding Prediction Errors in Learned Dynamics Modelscited by 47
2021Augmented World Models Facilitate Zero-Shot Dynamics Generalization From a Single Offline Environmentcited by 41
2021Adaptive-Control-Oriented Meta-Learning for Nonlinear Systemscited by 38
2022Skill-based Meta-Reinforcement Learningcited by 36
2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
2023Multi-agent Reinforcement Learning: A Comprehensive Surveycited by 30

38 more

2021Model Predictive Actor-Critic: Accelerating Robot Skill Acquisition with Deep Reinforcement Learningcited by 25
2021Explainable Reinforcement Learning for Broad-XAI: A Conceptual Framework and Surveycited by 24
2021A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learningcited by 24
2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
2021Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Criticcited by 23
2021Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoirescited by 20
2022Control-oriented meta-learningcited by 17
2023Interpretable Reward Redistribution in Reinforcement Learning: A Causal Approachcited by 17
2023Distributionally Robust Model-based Reinforcement Learning with Large State Spacescited by 17
2021Minimax Model Learningcited by 16
2022Training Efficient Controllers via Analytic Policy Gradientcited by 16
2024Diffusion Spectral Representation for Reinforcement Learningcited by 15
2025Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robotscited by 15
2022Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learningcited by 14
2022A Unified Framework for Alternating Offline Model Training and Policy Learningcited by 14
2022Latent Variable Representation for Reinforcement Learningcited by 14
2021High-Accuracy Model-Based Reinforcement Learning, a Surveycited by 13
2021On Effective Scheduling of Model-based Reinforcement Learningcited by 13
2024MOTO: Offline Pre-training to Online Fine-tuning for Model-based Robot Learningcited by 13
2024MAMBA: an Effective World Model Approach for Meta-Reinforcement Learningcited by 13
2025Distilling Reinforcement Learning Algorithms for In-Context Model-Based Planningcited by 12
2021PerSim: Data-Efficient Offline Reinforcement Learning with Heterogeneous Agents via Personalized Simulatorscited by 10
2022Reinforcement Learning in Presence of Discrete Markovian Context Evolutioncited by 9
2023Sample Efficient Reinforcement Learning in Mixed Systems through Augmented Samples and Its Applications to Queueing Networkscited by 9
2023ContraBAR: Contrastive Bayes-Adaptive Deep RLcited by 9
2023Beyond Stationarity: Convergence Analysis of Stochastic Softmax Policy Gradient Methodscited by 9
2021Measuring Progress in Deep Reinforcement Learning Sample Efficiencycited by 8
2021Imaginary Hindsight Experience Replay: Curious Model-based Learning for Sparse Reward Taskscited by 8
2021ModelLight: Model-Based Meta-Reinforcement Learning for Traffic Signal Controlcited by 8
2024Acceleration for Deep Reinforcement Learning using Parallel and Distributed Computing: A Surveycited by 8
2022Gradient-Based Trajectory Optimization With Learned Dynamicscited by 7
2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 7
2021Robust Model-based Reinforcement Learning for Autonomous Greenhouse Controlcited by 6
2022Mingling Foresight with Imagination: Model-Based Cooperative Multi-Agent Reinforcement Learningcited by 6
2023The AI Revolution: Opportunities and Challenges for the Finance Sectorcited by 6
2023On Representation Complexity of Model-based and Model-free Reinforcement Learningcited by 6
2023Data-Efficient Task Generalization via Probabilistic Model-based Meta Reinforcement Learningcited by 6
2024Policy-shaped prediction: avoiding distractions in model-based reinforcement learningcited by 6

50 most cited of 99 All 99 (JSON)