mlsanityData to 7 Oct 2026

Calibrated Model-Based Deep Reinforcement Learning

arXiv 1906.08312 ↗First posted 19 Jun 2019cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

Estimates of predictive uncertainty are important for accurate model-based planning and reinforcement learning. However, predictive uncertainties---especially ones derived from modern deep learning systems---can be inaccurate and impose a bottleneck on performance. This paper explores which uncertainties are needed for model-based reinforcement learning and argues that good uncertainties must be calibrated, i.e. their probabilities should match empirical frequencies of predicted events. We describe a simple way to augment any model-based reinforcement learning agent with a calibrated model and show that doing so consistently improves planning, sample complexity, and exploration. On the textscHalfCheetah MuJoCo task, our system achieves state-of-the-art performance using 50% fewer samples than the current leading approach. Our findings suggest that calibration can improve the performance of model-based reinforcement learning with minimal computational and implementation overhead.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 10; 2022: 4; 2023: 7; 2024: 5; 2025: 2; 2026: 3.

31 papers.

Sort:

By area:

  1. 2021Offline Reinforcement Learning as One Big Sequence Modeling Problemcited by 636
  2. 2021Uncertainty Toolbox: an Open-Source Library for Assessing, Visualizing, and Improving Uncertainty Quantificationcited by 57
  3. 2024Cal-DPO: Calibrated Direct Preference Optimization for Language Model Alignmentcited by 50
  4. 2021PAC Prediction Sets Under Covariate Shiftcited by 42
  5. 2023The Alignment Ceiling: Objective Mismatch in Reinforcement Learning from Human Feedbackcited by 42
  6. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  7. 2024Cross-Domain Policy Adaptation by Capturing Representation Mismatchcited by 24
  8. 2023Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matchingcited by 17
  9. 2022Modular Conformal Calibrationcited by 15
  10. 2021Combining Pessimism with Optimism for Robust and Efficient Model-Based Deep Reinforcement Learningcited by 14
  11. 2021Model-based micro-data reinforcement learning: what are the crucial model properties and which model to choose?cited by 11
  12. 2021Online Calibrated and Conformal Prediction Improves Bayesian Optimizationcited by 10
19 more
  1. 2021Safety Enhancement for Deep Reinforcement Learning in Autonomous Separation Assurancecited by 8
  2. 2022Towards PAC Multi-Object Detection and Trackingcited by 8
  3. 2024Calibration of Continual Learning Modelscited by 6
  4. 2024Zero-shot Model-based Reinforcement Learning using Large Language Modelscited by 6
  5. 2023Model-based Offline Reinforcement Learning with Local Misspecificationcited by 5
  6. 2024Calibrated Probabilistic Forecasts for Arbitrary Sequencescited by 4
  7. 2021DROMO: Distributionally Robust Offline Model-based Policy Optimizationcited by 3
  8. 2022Approaching sales forecasting using recurrent neural networks and transformerscited by 3
  9. 2022A Verification Framework for Certifying Learning-Based Safety-Critical Aviation Systemscited by 3
  10. 2023Calibrated Regression Against An Adversary Without Regretcited by 3
  11. 2023Calibrated and Conformal Propensity Scores for Causal Effect Estimationcited by 3
  12. 2023Safe Model-Based Multi-Agent Mean-Field Reinforcement Learningcited by 3
  13. 2025h-calibration: Rethinking Classifier Recalibration with Probabilistic Error-Bounded Objectivecited by 2
  14. 2025Bellman Calibration for V-Learning in Offline Reinforcement Learningcited by 2
  15. 2026Calibrated Multivariate Distributional Regression with Pre-Rank Regularizationcited by 2
  16. 2021MBDP: A Model-based Approach to Achieve both Robustness and Sample Efficiency via Double Dropout Planningcited by 0
  17. 2023Meta-learning to Calibrate Gaussian Processes with Deep Kernels for Regression Uncertainty Estimationcited by 0
  18. 2026Optimistic Policy Learning under Pessimistic Adversaries with Regret and Violation Guaranteescited by 0
  19. 2026Lucid Dreaming for World Models: Learning to Doubt Imagination and Decide by Trustcited by 0