mlsanityData to 7 Oct 2026

Benchmarking Model-Based Reinforcement Learning

arXiv 1907.02057 ↗First posted 3 Jul 2019cs.LG cs.AI cs.ROcited by 158

Not on the map. Why?

Abstract (arXiv)

Model-based reinforcement learning (MBRL) is widely seen as having the potential to be significantly more sample efficient than model-free RL. However, research in model-based RL has not been very standardized. It is fairly common for authors to experiment with self-designed environments, and there are several separate lines of research, which are sometimes closed-sourced or not reproducible. Accordingly, it is an open question how these various existing MBRL algorithms perform relative to each other. To facilitate research in MBRL, in this paper we gather a wide collection of MBRL algorithms and propose over 18 benchmarking environments specially designed for MBRL. We benchmark these algorithms with unified problem settings, including noisy environments. Beyond cataloguing performance, we explore and unify the underlying algorithmic differences across MBRL algorithms. We characterize three key research challenges for future MBRL research: the dynamics bottleneck, the planning horizon dilemma, and the early-termination dilemma. Finally, to maximally facilitate future research on MBRL, we open-source our benchmark in http://www.cs.toronto.edu/~tingwuwang/mbrl.html.

Key facts. Cited by 158 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jul 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 1; 2020: 4; 2021: 41; 2022: 35; 2023: 27; 2024: 23; 2025: 21; 2026: 6.

158 papers.

Sort:

By area:

  1. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  2. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 699
  3. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  4. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 582
  5. 2021Randomized Ensembled Double Q-Learning: Learning Fast Without a Modelcited by 278
  6. 2021What Matters for Adversarial Imitation Learning?cited by 70
  7. 2022A Survey on Model-based Reinforcement Learningcited by 65
  8. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  9. 2024Bridging State and History Representations: Understanding Self-Predictive RLcited by 52
  10. 2022VRL3: A Data-Driven Framework for Visual Deep Reinforcement Learningcited by 51
  11. 2022Making Linear MDPs Practical via Contrastive Representation Learningcited by 49
  12. 2022Investigating Compounding Prediction Errors in Learned Dynamics Modelscited by 47
38 more
  1. 2021MobILE: Model-Based Imitation Learning From Observation Alonecited by 44
  2. 2021safe-control-gym: a Unified Benchmark Suite for Safe Learning-based Control and Reinforcement Learning in Roboticscited by 42
  3. 2021MBRL-Lib: A Modular Library for Model-based Reinforcement Learningcited by 41
  4. 2022Physical Design using Differentiable Learned Simulatorscited by 37
  5. 2022Spectral Decomposition Representation for Reinforcement Learningcited by 35
  6. 2023World Models and Predictive Coding for Cognitive and Developmental Robotics: Frontiers and Challengescited by 35
  7. 2021Deep reinforcement learning in medical imaging: A literature reviewcited by 34
  8. 2022Generalizing Goal-Conditioned Reinforcement Learning with Variational Causal Reasoningcited by 33
  9. 2021Autoregressive Dynamics Models for Offline Policy Evaluation and Optimizationcited by 32
  10. 2024Diffusion World Model: Future Modeling Beyond Step-by-Step Rollout for Offline Reinforcement Learningcited by 31
  11. 2021Procedural Generalization by Planning with Self-Supervised World Modelscited by 30
  12. 2023Rethinking Closed-loop Training for Autonomous Drivingcited by 29
  13. 2022Simplifying Model-based RL: Learning Representations, Latent-space Models, and Policies with One Objectivecited by 28
  14. 2020Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learningcited by 27
  15. 2021A Consciousness-Inspired Planning Agent for Model-Based Reinforcement Learningcited by 27
  16. 2022Model-based Safe Deep Reinforcement Learning via a Constrained Proximal Policy Optimization Algorithmcited by 27
  17. 2023RL-ViGen: A Reinforcement Learning Benchmark for Visual Generalizationcited by 27
  18. 2022Sample Efficient Deep Reinforcement Learning via Uncertainty Estimationcited by 26
  19. 2023CCIL: Continuity-based Data Augmentation for Corrective Imitation Learningcited by 26
  20. 2022Deep Koopman Operator with Control for Nonlinear Systemscited by 25
  21. 2021Model-based Multi-agent Policy Optimization with Adaptive Opponent-wise Rolloutscited by 24
  22. 2021A Free Lunch from the Noise: Provable and Practical Exploration for Representation Learningcited by 24
  23. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  24. 2021Sample-Efficient Reinforcement Learning via Conservative Model-Based Actor-Criticcited by 23
  25. 2025AdaWM: Adaptive World Model based Planning for Autonomous Drivingcited by 23
  26. 2021Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Controlcited by 22
  27. 2022SAM-RL: Sensing-Aware Model-Based Reinforcement Learning via Differentiable Physics-Based Simulation and Renderingcited by 22
  28. 2023Causal Reinforcement Learning: A Surveycited by 22
  29. 2021Dynamics-Aware Quality-Diversity for Efficient Learning of Skill Repertoirescited by 20
  30. 2023Seizing Serendipity: Exploiting the Value of Past Success in Off-Policy Actor-Criticcited by 20
  31. 2025Learning Transformer-based World Models with Contrastive Predictive Codingcited by 20
  32. 2025Synthesis of Model Predictive Control and Reinforcement Learning: Survey and Classificationcited by 19
  33. 2021Learning Modular Robot Control Policiescited by 18
  34. 2022Model-based Multi-agent Reinforcement Learning: Recent Progress and Prospectscited by 17
  35. 2021MAMBPO: Sample-efficient multi-robot reinforcement learning using learned world modelscited by 16
  36. 2021SAGCI-System: Towards Sample-Efficient, Generalizable, Compositional, and Incremental Robot Learningcited by 16
  37. 2021Co-Adaptation of Algorithmic and Implementational Innovations in Inference-based Deep Reinforcement Learningcited by 15
  38. 2024Diffusion Spectral Representation for Reinforcement Learningcited by 15

50 most cited of 158 All 158 (JSON)