mlsanityData to 14 Oct 2026

Look Before You Leap: Safe Model-Based Reinforcement Learning with Human Intervention

arXiv 2111.05819 ↗First posted 10 Nov 2021cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Safety has become one of the main challenges of applying deep reinforcement learning to real world systems. Currently, the incorporation of external knowledge such as human oversight is the only means to prevent the agent from visiting the catastrophic state. In this paper, we propose MBHI, a novel framework for safe model-based reinforcement learning, which ensures safety in the state-level and can effectively avoid both "local" and "non-local" catastrophes. An ensemble of supervised learners are trained in MBHI to imitate human blocking decisions. Similar to human decision-making process, MBHI will roll out an imagined trajectory in the dynamics model before executing actions to the environment, and estimate its safety. When the imagination encounters a catastrophe, MBHI will block the current action and use an efficient MPC method to output a safety policy. We evaluate our method on several safety tasks, and the results show that MBHI achieved better performance in terms of sample efficiency and number of catastrophes compared to the baselines.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 33 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

28 of 33 references.

References · 28

  1. 2015Continuous control with deep reinforcement learningcited by 3,407
  2. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  3. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  4. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
  5. 2015Prioritized Experience Replaycited by 1,099
  6. 2017Constrained Policy Optimizationcited by 983
  7. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  8. 2019When to Trust Your Model: Model-Based Policy Optimizationcited by 773
  9. 2017Safe Reinforcement Learning via Shieldingcited by 449
  10. 2017Measuring Catastrophic Forgetting in Neural Networkscited by 360
  11. 2015Risk-Constrained Reinforcement Learning with Percentile Risk Criteriacited by 337
  12. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 319
16 more
  1. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  2. 2018Safe Exploration in Continuous Action Spacescited by 247
  3. 2018Model-Ensemble Trust-Region Policy Optimizationcited by 219
  4. 2018Sample-Efficient Reinforcement Learning with Stochastic Ensemble Value Expansioncited by 168
  5. 2018Model-Based Value Estimation for Efficient Model-Free Reinforcement Learningcited by 156
  6. 2017Trial without Error: Towards Safe Reinforcement Learning via Human Interventioncited by 103
  7. 2017Uncertainty-Aware Reinforcement Learning for Collision Avoidancecited by 88
  8. 2018Safe Reinforcement Learning with Model Uncertainty Estimatescited by 72
  9. 2017OptLayer - Practical Constrained Optimization for Deep Reinforcement Learning in the Real Worldcited by 70
  10. 2020Cautious Adaptation For Reinforcement Learning in Safety-Critical Settingscited by 42
  11. 2020Trust the Model When It Is Confident: Masked Model-based Actor-Criticcited by 36
  12. 2020Ready Policy One: World Building Through Active Learningcited by 33
  13. 2020SAMBA: Safe Model-Based & Active Reinforcement Learningcited by 30
  14. 2014Policy Gradients for CVaR-Constrained MDPscited by 21
  15. 2019Improving Safety in Reinforcement Learning Using Model-Based Architectures and Human Interventioncited by 8
  16. 2019Safety-Guided Deep Reinforcement Learning via Online Gaussian Process Estimationcited by 5

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 0; 2025: 3.

5 papers.

  1. 2023Efficient Preference-Based Reinforcement Learning Using Learned Dynamics Modelscited by 21
  2. 2025Learning from Active Human Involvement through Proxy Value Propagationcited by 19
  3. 2025Data-Efficient Learning from Human Interventions for Mobile Robotscited by 4
  4. 2023Agent-Specific Effects: A Causal Effect Propagation Analysis in Multi-Agent MDPscited by 1
  5. 2025Safe But Not Sorry: Reducing Over-Conservatism in Safety Critics via Uncertainty-Aware Modulationcited by 0