mlsanityData to 7 Oct 2026

Stable Inverse Reinforcement Learning: Policies from Control Lyapunov Landscapes

arXiv 2405.08756 ↗First posted 14 May 2024eess.SY cs.LG cs.SYcited by 2

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Learning from expert demonstrations to flexibly program an autonomous system with complex behaviors or to predict an agent's behavior is a powerful tool, especially in collaborative control settings. A common method to solve this problem is inverse reinforcement learning (IRL), where the observed agent, e.g., a human demonstrator, is assumed to behave according to the optimization of an intrinsic cost function that reflects its intent and informs its control actions. While the framework is expressive, it is also computationally demanding and generally lacks convergence guarantees. We therefore propose a novel, stability-certified IRL approach by reformulating the cost function inference problem to learning control Lyapunov functions (CLF) from demonstrations data. By additionally exploiting closed-form expressions for associated control policies, we are able to efficiently search the space of CLFs by observing the attractor landscape of the induced dynamics. For the construction of the inverse optimal CLFs, we use a Sum of Squares and formulate a convex optimization problem. We present a theoretical analysis of the optimality properties provided by the CLF and evaluate our approach using both simulated and real-world data.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 64 references that are in the map. First posted on arXiv 14 May 2024.

Builds on

10 of 64 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Inverse Reinforcement Learning: A Control Lyapunov Approach9 Apr
  2. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 8

  1. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  2. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  3. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  4. 2019Nonlinear System Identification: A User-Oriented Roadmapcited by 55
  5. 2020Learning Dynamical Systems with Side Informationcited by 19
  6. 2022Safe Control Synthesis with Uncertain Dynamics and Constraintscited by 7
  7. 2020Shape-Constrained Regression using Sum of Squares Polynomialscited by 5
  8. 2023Feasibility and Regularity Analysis of Safe Stabilizing Controllers under Uncertaintycited by 3

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Trust Region Reward Optimization and Proximal Inverse Reward Optimization Algorithmcited by 1
  2. 2025Learning Geometrically-Informed Lyapunov Functions with Deep Diffeomorphic RBF Networkscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021Inverse Reinforcement Learning in a Continuous State Space with Formal Guarantees
  2. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  3. 2025Offline Reinforcement Learning via Inverse Optimization
  4. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  5. 2024Inverse Reinforcement Learning from Non-Stationary Learning Agents
  6. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  7. 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
  8. 2021A Generalised Inverse Reinforcement Learning Framework