# Stable Inverse Reinforcement Learning: Policies from Control Lyapunov Landscapes

arXiv: https://arxiv.org/abs/2405.08756 · HTML: https://mlsanity.com/paper/2405.08756/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 10 of its 64 references that are in the map.
- First posted on arXiv 14 May 2024.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)
- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 10 of 10)

- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress](https://mlsanity.com/paper/1806.06877/) (2018)
- [A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Models](https://mlsanity.com/paper/1611.03852/) (2016)
- [Nonlinear System Identification: A User-Oriented Roadmap](https://mlsanity.com/paper/1902.00683/) (2019)
- [Learning Dynamical Systems with Side Information](https://mlsanity.com/paper/2008.10135/) (2020)
- [Safe Control Synthesis with Uncertain Dynamics and Constraints](https://mlsanity.com/paper/2202.09557/) (2022)
- [Shape-Constrained Regression using Sum of Squares Polynomials](https://mlsanity.com/paper/2004.03853/) (2020)
- [Inverse Reinforcement Learning: A Control Lyapunov Approach](https://mlsanity.com/paper/2104.04483/) (2021)
- [Feasibility and Regularity Analysis of Safe Stabilizing Controllers under Uncertainty](https://mlsanity.com/paper/2301.04603/) (2023)

## Built on by (most cited first, 3 of 3)

- [A Review On Safe Reinforcement Learning Using Lyapunov and Barrier Functions](https://mlsanity.com/paper/2508.09128/) (2025)
- [Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithm](https://mlsanity.com/paper/2509.23135/) (2025)
- [Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networks](https://mlsanity.com/paper/2504.02607/) (2025)

## Abstract (arXiv)

Learning from expert demonstrations to flexibly program an autonomous system with complex behaviors or to predict an agent's behavior is a powerful tool, especially in collaborative control settings. A common method to solve this problem is inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to the optimization of an intrinsic cost function that reflects its intent and informs its control actions. While the framework is expressive, it is also computationally demanding and generally lacks convergence guarantees. We therefore propose a novel, stability-certified IRL approach by reformulating the cost function inference problem to learning control Lyapunov functions (CLF) from demonstrations data. By additionally exploiting closed-form expressions for associated control policies, we are able to efficiently search the space of CLFs by observing the attractor landscape of the induced dynamics. For the construction of the inverse optimal CLFs, we use a Sum of Squares and formulate a convex optimization problem. We present a theoretical analysis of the optimality properties provided by the CLF and evaluate our approach using both simulated and real-world data.