mlsanityData to 14 Oct 2026

Safe Model-based Reinforcement Learning with Stability Guarantees

arXiv 1705.08551 ↗First posted 23 May 2017stat.ML cs.AI cs.LGcited by 366

Not on the map. Why?

Cited by 97 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Reinforcement learning is a powerful paradigm for learning optimal policies from experimental data. However, to find optimal policies, most reinforcement learning algorithms explore all possible actions, which may be harmful for real-world systems. As a consequence, learning algorithms are rarely applied on safety-critical systems in the real world. In this paper, we present a learning algorithm that explicitly considers safety, defined in terms of stability guarantees. Specifically, we extend control-theoretic results on Lyapunov stability verification and show how to use statistical models of the dynamics to obtain high-performance control policies with provable stability certificates. Moreover, under additional regularity assumptions in terms of a Gaussian process prior, we prove that one can effectively and safely collect data in order to learn about the dynamics and thus both improve control performance and expand the safe region of the state space. In our experiments, we show how the resulting algorithm can safely optimize a neural network policy on a simulated inverted pendulum, without the pendulum ever falling down.

Key facts. Cited by 366 papers in the map (data through 14 Oct 2026). First posted on arXiv 23 May 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 2; 2021: 86; 2022: 65; 2023: 58; 2024: 50; 2025: 47; 2026: 57.

366 papers.

Sort:

By area:

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 680
  3. 2021Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learningcited by 371
  4. 2022A Review of Safe Reinforcement Learning: Methods, Theory and Applicationscited by 181
  5. 2022TransDreamer: Reinforcement Learning with Transformer World Modelscited by 143
  6. 2018EnsembleDAgger: A Bayesian Approach to Safe Imitation Learningcited by 100
  7. 2023The Safety Filter: A Unified View of Safety-Critical Control in Autonomous Systemscited by 88
  8. 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 80
  9. 2021Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learningcited by 79
  10. 2022Saute RL: Almost Surely Safe Reinforcement Learning Using State Augmentationcited by 69
  11. 2022Enforcing Hard Constraints with Soft Barriers: Safe Reinforcement Learning in Unknown Stochastic Environmentscited by 60
  12. 2023Language Models, Agent Models, and World Models: The LAW for Machine Reasoning and Planningcited by 60
38 more
  1. 2021Learning to Synthesize Programs as Interpretable and Generalizable Policiescited by 59
  2. 2021Safe Reinforcement Learning Using Robust Control Barrier Functionscited by 59
  3. 2022Constrained Policy Optimization via Bayesian World Modelscited by 57
  4. 2023State-wise Safe Reinforcement Learning: A Surveycited by 55
  5. 2021Reinforcement Learning for Selective Key Applications in Power Systems: Recent Advances and Future Challengescited by 50
  6. 2022Hierarchical Planning Through Goal-Conditioned Offline Reinforcement Learningcited by 50
  7. 2021Practical and Rigorous Uncertainty Bounds for Gaussian Process Regressioncited by 49
  8. 2024Enhancing LLM Safety via Constrained Direct Preference Optimizationcited by 49
  9. 2021Safe Reinforcement Learning Using Advantage-Based Interventioncited by 47
  10. 2021Contraction Theory for Nonlinear Stability Analysis and Learning-based Control: A Tutorial Overviewcited by 45
  11. 2024A Survey of Constraint Formulations in Safe Reinforcement Learningcited by 44
  12. 2022Neural Lyapunov Control of Unknown Nonlinear Systems with Stability Guaranteescited by 40
  13. 2021How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Reviewcited by 39
  14. 2021Learning Barrier Certificates: Towards Safe Reinforcement Learning with Zero Training-time Violationscited by 39
  15. 2022On the Robustness of Safe Reinforcement Learning under Observational Perturbationscited by 38
  16. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 36
  17. 2021Safe Reinforcement Learning with Linear Function Approximationcited by 35
  18. 2022Safe Reinforcement Learning for Legged Locomotioncited by 35
  19. 2022Provably Safe Reinforcement Learning: Conceptual Analysis, Survey, and Benchmarkingcited by 34
  20. 2021Autonomous Reinforcement Learning via Subgoal Curriculacited by 33
  21. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  22. 2023Optimistic Active Exploration of Dynamical Systemscited by 32
  23. 2021Enforcing Policy Feasibility Constraints through Differentiable Projection for Energy Optimizationcited by 31
  24. 2022Towards Safe Reinforcement Learning with a Safety Editor Policycited by 30
  25. 2022Sustainable Online Reinforcement Learning for Auto-biddingcited by 29
  26. 2022Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Controlcited by 28
  27. 2023Physics-Informed Machine Learning for Modeling and Control of Dynamical Systemscited by 28
  28. 2021GoSafe: Globally Optimal Safe Robot Learningcited by 27
  29. 2021Safe Pontryagin Differentiable Programmingcited by 26
  30. 2021Stability Verification in Stochastic Control Systems via Neural Network Supermartingalescited by 26
  31. 2023Neural Lyapunov Control for Discrete-Time Systemscited by 23
  32. 2024ActSafe: Active Exploration with Safety Constraints for Reinforcement Learningcited by 23
  33. 2022Prediction Sets Adaptive to Unknown Covariate Shiftcited by 22
  34. 2021Safe Reinforcement Learning with Chance-constrained Model Predictive Controlcited by 20
  35. 2022Language Control Diffusion: Efficiently Scaling through Space, Time, and Taskscited by 20
  36. 2021Value Iteration in Continuous Actions, States and Timecited by 18
  37. 2023Neural Operators of Backstepping Controller and Observer Gain Functions for Reaction-Diffusion PDEscited by 18
  38. 2021Safe Continuous Control with Constrained Model-Based Policy Optimizationcited by 17

50 most cited of 366 All 366 (JSON)