# Online Observer-Based Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2011.02057 · HTML: https://mlsanity.com/paper/2011.02057/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 9 of its 31 references that are in the map.
- First posted on arXiv 3 Nov 2020.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 9 of 9)

- [Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations](https://mlsanity.com/paper/1904.06387/) (2019)
- [Maximum Entropy Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/1507.04888/) (2015)
- [Dialogue Generation: From Imitation Learning to Inverse Reinforcement Learning](https://mlsanity.com/paper/1812.03509/) (2018)
- [Inverse Reinforcement Learning in Swarm Systems](https://mlsanity.com/paper/1602.05450/) (2016)
- [Competitive Multi-agent Inverse Reinforcement Learning with Sub-optimal Demonstrations](https://mlsanity.com/paper/1801.02124/) (2018)
- [Online inverse reinforcement learning with unknown disturbances](https://mlsanity.com/paper/2003.03912/) (2020)
- [Inverse reinforcement learning in continuous time and space](https://mlsanity.com/paper/1801.07663/) (2018)
- [Online Output-Feedback Parameter and State Estimation for Second Order Linear Systems](https://mlsanity.com/paper/1609.05879/) (2016)
- [Online inverse reinforcement learning with limited data](https://mlsanity.com/paper/2008.08972/) (2020)

## Built on by (most cited first, 4 of 4)

- [Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games](https://mlsanity.com/paper/2301.01997/) (2023)
- [Nonuniqueness and Convergence to Equivalent Solutions in Observer-based Inverse Reinforcement Learning](https://mlsanity.com/paper/2210.16299/) (2022)
- [Pilot Performance modeling via observer-based inverse reinforcement learning](https://mlsanity.com/paper/2307.13150/) (2023)
- [Recursive Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2504.13241/) (2025)

## Abstract (arXiv)

In this paper, a novel approach to the output-feedback inverse reinforcement learning (IRL) problem is developed by casting the IRL problem, for linear systems with quadratic cost functions, as a state estimation problem. Two observer-based techniques for IRL are developed, including a novel observer method that re-uses previous state estimates via history stacks. Theoretical guarantees for convergence and robustness are established under appropriate excitation conditions. Simulations demonstrate the performance of the developed observers and filters under noisy and noise-free measurements.