mlsanityData to 13 Oct 2026

Bounding-Box Inference for Error-Aware Model-Based Reinforcement Learning

arXiv 2406.16006 ↗First posted 23 Jun 2024cs.LG cs.AIcited by 1

On the map: Deep reinforcement learning methods → RL theory and sample complexity

Abstract (arXiv)

In model-based reinforcement learning, simulated experiences from the learned model are often treated as equivalent to experience from the real environment. However, when the model is inaccurate, it can catastrophically interfere with policy learning. Alternatively, the agent might learn about the model's accuracy and selectively use it only when it can provide reliable predictions. We empirically explore model uncertainty measures for selective planning and show that best results require distribution insensitive inference to estimate the uncertainty over model-based updates. To that end, we propose and evaluate bounding-box inference, which operates on bounding-boxes around sets of possible states and other quantities. We find that bounding-box inference can reliably support effective selective planning.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in RL theory and sample complexity, one of 15 directions in Deep reinforcement learning methods. Builds on 14 of its 26 references that are in the map. First posted on arXiv 23 Jun 2024.

Builds on

14 of 26 references.

References · 14

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,093
  2. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,334
  3. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 756
  4. 2018Implicit Quantile Networks for Distributional Reinforcement Learningcited by 363
  5. 2018Randomized Prior Functions for Deep Reinforcement Learningcited by 219
  6. 2015Action-Conditional Video Prediction using Deep Networks in Atari Gamescited by 172
  7. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 167
  8. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 152
  9. 2020The Value Equivalence Principle for Model-Based Reinforcement Learningcited by 90
  10. 2020Dreaming: Model-based Reinforcement Learning by Latent Imagination without Reconstructioncited by 66
  11. 2016Self-Correcting Models for Model-Based Reinforcement Learningcited by 42
  12. 2020Selective Dyna-style Planning Under Limited Model Capacitycited by 30
2 more
  1. 2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
  2. 2019Planning with Expectation Modelscited by 12

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024MAD-TD: Model-Augmented Data stabilizes High Update Ratio RLcited by 16