mlsanityData to 14 Oct 2026

All Models are Wrong, Knowing Where is Useful: On Model Uncertainty in Reinforcement Learning

arXiv 2606.01363 ↗First posted 31 May 2026cs.LG cs.SY eess.SYcited by 0

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

Model-based reinforcement learning (MBRL) infers information about the environment from a learned dynamics model and bears the potential to address open problems such as data efficient and safe learning in robotics. However, inaccuracies of the learned dynamics model are typically exploited by the agent, substantially hampering the capabilities of MBRL methods. We present a framework for dealing with inaccuracies of probabilistic models through targeted handling of uncertainty that effectively mitigates model exploitation. We present recent successes in learning directly on hardware and safe exploration, and discuss future directions for uncertainty-aware MBRL.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 37 of its 46 references that are in the map. First posted on arXiv 31 May 2026.

Builds on

37 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025On Rollouts in Model-Based Reinforcement Learning28 Jan
  2. 2024Trust the Model Where It Trusts Itself -- Model-Based Actor-Critic with Uncertainty-Aware Rollout Adaption29 May

Other references · 35

  1. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  2. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  3. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  4. 2017Constrained Policy Optimizationcited by 983
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  6. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  7. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  8. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  9. 2023TD-MPC2: Scalable, Robust World Models for Continuous Controlcited by 512
  10. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  11. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  12. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
23 more
  1. 2024Deep Reinforcement Learning for Robotics: A Survey of Real-World Successescited by 176
  2. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  3. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  4. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  5. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guaranteescited by 140
  6. 2018Learning-based Model Predictive Control for Safe Explorationcited by 120
  7. 2016Learning and Policy Search in Stochastic Dynamical Systems with Bayesian Neural Networkscited by 49
  8. 2018Linear model predictive safety certification for learning-based controlcited by 37
  9. 2024MaxInfoRL: Boosting exploration in reinforcement learning through information gain maximizationcited by 30
  10. 2019Actively Learning Gaussian Process Dynamicscited by 26
  11. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 23
  12. 2024Safety Filtering While Training: Improving the Performance and Sample Efficiency of Reinforcement Learning Agentscited by 17
  13. 2025Uncertainty-Aware Robotic World Model Makes Offline Model-Based Reinforcement Learning Work on Real Robotscited by 17
  14. 2022On Uncertainty in Deep State Space Models for Model-Based Reinforcement Learningcited by 13
  15. 2024Safety Filters for Black-Box Dynamical Systems by Learning Discriminating Hyperplanescited by 12
  16. 2023Approximate non-linear model predictive control with safety-augmented neural networkscited by 9
  17. 2023Data-efficient Deep Reinforcement Learning for Vehicle Trajectory Controlcited by 4
  18. 2024Reinforcement Learning with Ensemble Model Predictive Safety Certificationcited by 4
  19. 2025The Mini Wheelbot: A Testbed for Learning-based Balancing, Flips, and Articulated Drivingcited by 4
  20. 2023Progressive Transfer Learning for Dexterous In-Hand Manipulation with Multi-Fingered Anthropomorphic Handcited by 3
  21. 2026Dyna-Style Safety Augmented Reinforcement Learning: Staying Safe in the Face of Uncertaintycited by 1
  22. 2026Uncertainty-Aware Predictive Safety Filters for Probabilistic Neural Network Dynamicscited by 1
  23. 2026Learning to Race in Minutes: Infoprop Dyna on the Mini Wheelbotcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Look Before Leap: Look-Ahead Planning with Uncertainty in Reinforcement Learning
  2. 2021Revisiting Design Choices in Offline Model-Based Reinforcement Learning
  3. 2026Sampling-Based Safe Reinforcement Learning
  4. 2024Safe Exploration Using Bayesian World Models and Log-Barrier Optimization
  5. 2026CAPSULE: Control-Theoretic Action Perturbations for Safe Uncertainty-Aware Reinforcement Learning
  6. 2024Offline Model-Based Reinforcement Learning with Anti-Exploration
  7. 2021Safe Continuous Control with Constrained Model-Based Policy Optimization
  8. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidance