# Inverse reinforcement learning by expert imitation for the stochastic linear-quadratic optimal control problem

arXiv: https://arxiv.org/abs/2405.17085 · HTML: https://mlsanity.com/paper/2405.17085/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 47 references that are in the map.
- First posted on arXiv 27 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 9 of 9)

- [Sequential Anomaly Detection using Inverse Reinforcement Learning](https://mlsanity.com/paper/2004.10398/) (2020)
- [Robust Policy Iteration for Continuous-time Linear Quadratic Regulation](https://mlsanity.com/paper/2005.09528/) (2020)
- [Stochastic Linear Quadratic Optimal Control Problem: A Reinforcement Learning Method](https://mlsanity.com/paper/2011.14516/) (2020)
- [Reinforcement Learning for Adaptive Optimal Stationary Control of Linear Stochastic Systems](https://mlsanity.com/paper/2107.07788/) (2021)
- [Continuous-Time Robust Dynamic Programming](https://mlsanity.com/paper/1809.05867/) (2018)
- [Inverse stochastic optimal controls](https://mlsanity.com/paper/2005.11485/) (2020)
- [Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games](https://mlsanity.com/paper/2301.01997/) (2023)
- [Inverse optimal control for averaged cost per stage linear quadratic regulators](https://mlsanity.com/paper/2305.15332/) (2023)
- [Inverse Stochastic Optimal Control for Linear-Quadratic Gaussian and Linear-Quadratic Sensorimotor Control Models](https://mlsanity.com/paper/2210.17265/) (2022)

## Built on by (most cited first, 3 of 3)

- [SCaLE: Switching Cost aware Learning and Exploration](https://mlsanity.com/paper/2601.09042/) (2026)
- [Inverse Optimal Control for Linear Quadratic Problem with Poisson Jumps: Model-Free Inverse Reinforcement Learning Approaches](https://mlsanity.com/paper/2607.03169/) (2026)
- [Inverse reinforcement learning for indefinite mean-field social optimization with multiplicative noise](https://mlsanity.com/paper/2608.07252/) (2026)

## Abstract (arXiv)

This article studies inverse reinforcement learning (IRL) for the stochastic linear-quadratic optimal control problem, where two agents are considered. A learner agent does not know the expert agent's performance cost function, but it imitates the behavior of the expert agent by constructing an underlying cost function that obtains the same optimal feedback control as the expert's. We first develop a model-based IRL algorithm, which consists of a policy correction and a policy update from the policy iteration in reinforcement learning, as well as a cost function weight reconstruction based on the inverse optimal control. Then, under this scheme, we propose a model-free off-policy IRL algorithm, which does not need to know or identify the system and only needs to collect the behavior data of the expert agent and learner agent once during the iteration process. Moreover, the proofs of the algorithm's convergence, stability, and non-unique solutions are given. Finally, a simulation example is provided to verify the effectiveness of the proposed algorithm.