mlsanityData to 14 Oct 2026

Lyapunov Density Models: Constraining Distribution Shift in Learning-Based Control

arXiv 2206.10524 ↗First posted 21 Jun 2022cs.LG cs.SY eess.SYcited by 28

Not on the map. Why?

Abstract (arXiv)

Learned models and policies can generalize effectively when evaluated within the distribution of the training data, but can produce unpredictable and erroneous outputs on out-of-distribution inputs. In order to avoid distribution shift when deploying learning-based control algorithms, we seek a mechanism to constrain the agent to states and actions that resemble those that it was trained on. In control theory, Lyapunov stability and control-invariant sets allow us to make guarantees about controllers that stabilize the system around specific states, while in machine learning, density models allow us to estimate the training data distribution. Can we combine these two concepts, producing learning-based control algorithms that constrain the system to in-distribution states using only in-distribution actions? In this work, we propose to do this by combining concepts from Lyapunov stability and density estimation, introducing Lyapunov density models: a generalization of control Lyapunov functions and density models that provides guarantees on an agent's ability to stay in-distribution over its entire trajectory.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 21 Jun 2022.

Builds on

27 of 42 references.

References · 27

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  4. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  5. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 774
  6. 2018QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulationcited by 743
  7. 2019Solving Rubik's Cube with a Robot Handcited by 734
  8. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  9. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  10. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  11. 2019Neural Spline Flowscited by 596
  12. 2020MOReL : Model-Based Offline Reinforcement Learningcited by 590
15 more
  1. 2017Safe Model-based Reinforcement Learning with Stability Guaranteescited by 366
  2. 2019End-to-End Safe Reinforcement Learning through Barrier Functions for Safety-Critical Continuous Control Taskscited by 289
  3. 2016Control Barrier Function Based Quadratic Programs for Safety Critical Systemscited by 287
  4. 2017Hamilton-Jacobi Reachability: A Brief Overview and Recent Advancescited by 201
  5. 2017Safe Policy Improvement with Baseline Bootstrappingcited by 144
  6. 2020Neural Lyapunov Controlcited by 142
  7. 2021Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robotscited by 116
  8. 2015Gaussian Processes for Data-Efficient Learning in Robotics and Controlcited by 98
  9. 2020Learning Control Barrier Functions from Expert Demonstrationscited by 92
  10. 2018The Lyapunov Neural Network: Adaptive Stability Certification for Safe Learning of Dynamical Systemscited by 88
  11. 2021Lyapunov-stable neural-network controlcited by 66
  12. 2021On the Sample Complexity of Stability Constrained Imitation Learningcited by 38
  13. 2020Guaranteeing Safety of Learned Perception Modules via Measurement-Robust Control Barrier Functionscited by 33
  14. 2018Robustness to Out-of-Distribution Inputs via Task-Aware Generative Uncertaintycited by 14
  15. 2020Learning Dynamical Systems using Local Stability Priorscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 10; 2024: 3; 2025: 6; 2026: 8.

28 papers.

Sort:

By area:

  1. 2025Uncertainty-aware Latent Safety Filters for Avoiding Out-of-Distribution Failurescited by 33
  2. 2022Safe Model-Based Reinforcement Learning with an Uncertainty-Aware Reachability Certificatecited by 18
  3. 2023Fighting Uncertainty with Gradients: Offline Reinforcement Learning via Diffusion Score Matchingcited by 17
  4. 2023In-Distribution Barrier Functions: Self-Supervised Policy Filters that Avoid Out-of-Distribution Statescited by 16
  5. 2024SAFE-GIL: SAFEty Guided Imitation Learning for Robotic Systemscited by 14
  6. 2023A Survey of Demonstration Learningcited by 12
  7. 2023Predictable MDP Abstraction for Unsupervised Model-Based RLcited by 8
  8. 2025Elastic Motion Policy: An Adaptive Dynamical System for Robust and Efficient One-Shot Imitation Learningcited by 5
  9. 2025Learning Conservative Neural Control Barrier Functions from Offline Datacited by 5
  10. 2024Evaluating Defences against Unsafe Feedback in RLHFcited by 4
  11. 2025Safety Representations for Safer Policy Learningcited by 4
  12. 2026Pixels to Proofs: Probabilistically-Safe Latent World Model Control via Parallel Conformal Robust MPCcited by 4
16 more
  1. 2023Model Monitoring and Robustness of In-Use Machine Learning Models: Quantifying Data Distribution Shifts Using Population Stability Indexcited by 3
  2. 2023A Conservative Approach for Few-Shot Transfer in Off-Dynamics Reinforcement Learningcited by 3
  3. 2026Safety, Security, and Cognitive Risks in World Modelscited by 3
  4. 2023SeRO: Self-Supervised Reinforcement Learning for Recovery from Out-of-Distribution Situationscited by 2
  5. 2024Out-of-Distribution Adaptation in Offline RL: Counterfactual Reasoning via Causal Normalizing Flowscited by 2
  6. 2026SHAPO: Sharpness-Aware Policy Optimization for Safe Explorationcited by 2
  7. 2023Sim-to-Real Transfer of Adaptive Control Parameters for AUV Stabilization under Current Disturbancecited by 1
  8. 2025LaMOuR: Leveraging Language Models for Out-of-Distribution Recovery in Reinforcement Learningcited by 1
  9. 2025Distributionally Robust Imitation Learning: Layered Control Architecture for Certifiable Autonomycited by 1
  10. 2023DEFENDER: DTW-Based Episode Filtering Using Demonstrations for Enhancing RL Safetycited by 0
  11. 2023Dissipative Imitation Learning for Discrete Dynamic Output Feedback Control with Sparse Data Setscited by 0
  12. 2026Solving Parameter-Robust Avoid Problems with Unknown Feasibility using Reinforcement Learningcited by 0
  13. 2026Lyapunov-Guided Self-Alignment: Test-Time Adaptation for Offline Safe Reinforcement Learningcited by 0
  14. 2026Intrinsic Vicarious Conditioning for Deep Reinforcement Learningcited by 0
  15. 2026SafeExplorer: An Unbiased Policy Gradient for Reinforcement Learning with Recovery Interventionscited by 0
  16. 2026Learning-Based Motion Planning for Dynamic Environments: From Foundational Algorithms to Emerging Paradigmscited by 0