mlsanityData to 14 Oct 2026

Safe Deep Model-Based Reinforcement Learning with Lyapunov Functions

arXiv 2405.16184 ↗First posted 25 May 2024eess.SY cs.AI cs.LGcited by 2

On the map: Deep reinforcement learning methods → Safe and constrained RL

Abstract (arXiv)

Model-based Reinforcement Learning (MBRL) has shown many desirable properties for intelligent control tasks. However, satisfying safety and stability constraints during training and rollout remains an open question. We propose a new Model-based RL framework to enable efficient policy learning with unknown dynamics based on learning model predictive control (LMPC) framework with mathematically provable guarantees of stability. We introduce and explore a novel method for adding safety constraints for model-based RL during training and policy learning. The new stability-augmented framework consists of a neural-network-based learner that learns to construct a Lyapunov function, and a model-based RL agent to consistently complete the tasks while satisfying user-specified constraints given only sub-optimal demonstrations and sparse-cost feedback. We demonstrate the capability of the proposed framework through simulated experiments.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Safe and constrained RL, one of 15 directions in Deep reinforcement learning methods. Builds on 25 of its 37 references that are in the map. First posted on arXiv 25 May 2024.

Builds on

25 of 37 references.

References · 25

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,417
  3. 2017Constrained Policy Optimizationcited by 983
  4. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  5. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  6. 2017Neural Network Dynamics for Model-Based Deep Reinforcement Learning with Model-Free Fine-Tuningcited by 297
  7. 2018A Lyapunov-based Approach to Safe Reinforcement Learningcited by 297
  8. 2018Plan Online, Learn Offline: Efficient Learning and Exploration via Model-Based Controlcited by 113
  9. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  10. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  11. 2022TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulationcited by 47
  12. 2019Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entitiescited by 41
13 more
  1. 2016Learning Model Predictive Control for iterative tasks. A Data-Driven Control Frameworkcited by 39
  2. 2023FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projectioncited by 34
  3. 2015One-Shot Learning of Manipulation Skills with Online Dynamics Adaptation and Neural Network Priorscited by 26
  4. 2023DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classificationcited by 23
  5. 2020Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cablescited by 21
  6. 2021Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transformscited by 17
  7. 20206-DoF Grasp Planning using Fast 3D Reconstruction and Grasp Quality CNNcited by 13
  8. 2022ART/ATK: A research platform for assessing and mitigating the sim-to-real gap in robotics and autonomous vehicle engineeringcited by 10
  9. 2023APLA: Additional Perturbation for Latent Noise with Adversarial Training Enables Consistencycited by 9
  10. 2020ABC-LMPC: Safe Sample-Based Learning MPC for Stochastic Nonlinear Dynamical Systems with Adjustable Boundary Conditionscited by 8
  11. 2019Sample-Based Learning Model Predictive Control for Linear Uncertain Systemscited by 7
  12. 2016Sampling-based verification of Lyapunov's inequality for piecewise continuous nonlinear systemscited by 1
  13. 2019Safe Interactive Model-Based Learningcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2026WMAttack: Automated Attack Search for Adversarial Evaluation of World-Model Agentscited by 7
  2. 2025H2OFlow: Grounding Human-Object Affordances with 3D Generative Models and Dense Diffused Flowscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Safe Reinforcement Learning using Ideas from Model Predictive Control
  2. 2025Safe model-based Reinforcement Learning via Model Predictive Control and Control Barrier Functions
  3. 2024Sampling-based Safe Reinforcement Learning for Nonlinear Dynamical Systems
  4. 2025Model-free Reinforcement Learning for Model-based Control: Towards Safe, Interpretable and Sample-efficient Agents
  5. 2025A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions
  6. 2025Certifying Stability of Reinforcement Learning Policies using Generalized Lyapunov Functions
  7. 2018Algorithmic Framework for Model-based Deep Reinforcement Learning with Theoretical Guarantees
  8. 2024Robust Model Based Reinforcement Learning Using L1 Adaptive Control