mlsanityData to 14 Oct 2026

Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Models

arXiv 1805.12114 ↗First posted 30 May 2018cs.LG cs.AI cs.ROcited by 774

On the map: World models → Latent dynamics models and model-based RL

Abstract (arXiv)

Model-based reinforcement learning (RL) algorithms can attain excellent sample efficiency, but often lag behind the best model-free algorithms in terms of asymptotic performance. This is especially true with high-capacity parametric function approximators, such as deep networks. In this paper, we study how to bridge this gap, by employing uncertainty-aware dynamics models. We propose a new algorithm called probabilistic ensembles with trajectory sampling (PETS) that combines uncertainty-aware deep network dynamics models with sampling-based uncertainty propagation. Our comparison to state-of-the-art model-based and model-free deep RL algorithms shows that our approach matches the asymptotic performance of model-free algorithms on several challenging benchmark tasks, while requiring significantly fewer samples (e.g., 8 and 125 times fewer samples than Soft Actor Critic and Proximal Policy Optimization respectively on the half-cheetah task).

Key facts. Cited by 774 papers in the map (data through 14 Oct 2026). Placed in Latent dynamics models and model-based RL, one of 11 directions in World models. Builds on 4 of its 62 references that are in the map. First posted on arXiv 30 May 2018.

Builds on

4 of 62 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Model-Ensemble Trust-Region Policy Optimization28 Feb
  2. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuning8 Aug

Other references · 2

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2016Learning to Poke by Poking: Experiential Learning of Intuitive Physicscited by 154

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 2; 2019: 5; 2020: 12; 2021: 127; 2022: 122; 2023: 132; 2024: 115; 2025: 111; 2026: 148.

774 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,061
  5. 2020Mastering Atari with Discrete World Modelscited by 1,007
  6. 2021E(n) Equivariant Graph Neural Networkscited by 965
  7. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  8. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 644
  9. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  10. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
  11. 2022DayDreamer: World Models for Physical Robot Learningcited by 400
  12. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
38 more
  1. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  2. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  3. 2020Planning to Explore via Self-Supervised World Modelscited by 368
  4. 2021Mastering Atari Games with Limited Datacited by 237
  5. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 231
  6. 2022Masked World Models for Visual Controlcited by 170
  7. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  8. 2023Structured World Models from Human Videoscited by 147
  9. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
  10. 2021Actionable Models: Unsupervised Offline Reinforcement Learning of Robotic Skillscited by 125
  11. 2022Pessimistic Bootstrapping for Uncertainty-Driven Offline Reinforcement Learningcited by 125
  12. 2022On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networkscited by 120
  13. 2022RAMBO-RL: Robust Adversarial Model-Based Offline Reinforcement Learningcited by 115
  14. 2021Dropout Q-Functions for Doubly Efficient Reinforcement Learningcited by 114
  15. 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 114
  16. 2020Learning to Walk in the Real World with Minimal Human Effortcited by 111
  17. 2023Toward General-Purpose Robots via Foundation Models: A Survey and Meta-Analysiscited by 103
  18. 2020State-Only Imitation Learning for Dexterous Manipulationcited by 91
  19. 2023Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Modelscited by 91
  20. 2019Exploring Model-based Planning with Policy Networkscited by 90
  21. 2021Benchmarks for Deep Off-Policy Evaluationcited by 89
  22. 2021FitVid: Overfitting in Pixel-Level Video Predictioncited by 87
  23. 2023DREAMWALKER: Mental Planning for Continuous Vision-Language Navigationcited by 85
  24. 2022Does Zero-Shot Reinforcement Learning Exist?cited by 83
  25. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  26. 2021Residual Pathway Priors for Soft Equivariance Constraintscited by 75
  27. 2024Uncertainty-aware Reward Model: Teaching Reward Models to Know What is Unknowncited by 74
  28. 2020A Game Theoretic Framework for Model Based Reinforcement Learningcited by 73
  29. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  30. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  31. 2022Safe Reinforcement Learning by Imagining the Near Futurecited by 67
  32. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  33. 2022A Survey on Model-based Reinforcement Learningcited by 66
  34. 2022Causal Dynamics Learning for Task-Independent State Abstractioncited by 65
  35. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  36. 2024Self-Exploring Language Models: Active Preference Elicitation for Online Alignmentcited by 63
  37. 2021On the Importance of Hyperparameter Optimization for Model-based Reinforcement Learningcited by 59
  38. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 59

50 most cited of 774 All 774 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamics
  2. 2026Uncertainty-Aware Selection of Online Algorithms with Simulator Ensembles
  3. 2023Posterior Sampling for Deep Reinforcement Learning
  4. 2024Deterministic Uncertainty Propagation for Improved Model-Based Offline Reinforcement Learning
  5. 2024Model-Free Robust Reinforcement Learning with Sample Complexity Analysis
  6. 2022Incorporating Explicit Uncertainty Estimates into Deep Offline Reinforcement Learning
  7. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  8. 2023Balancing policy constraint and ensemble size in uncertainty-based offline reinforcement learning