# Online inverse reinforcement learning with unknown disturbances

arXiv: https://arxiv.org/abs/2003.03912 · HTML: https://mlsanity.com/paper/2003.03912/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 22 references that are in the map.
- First posted on arXiv 9 Mar 2020.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 2 of 2)

- [Inverse reinforcement learning in continuous time and space](https://mlsanity.com/paper/1801.07663/) (2018)
- [Online Simultaneous State and Parameter Estimation for Second-order Nonlinear Systems](https://mlsanity.com/paper/1703.07068/) (2017)

## Built on by (most cited first, 6 of 6)

- [A Comprehensive Survey on Inverse Constrained Reinforcement Learning: Definitions, Progress and Challenges](https://mlsanity.com/paper/2409.07569/) (2024)
- [Online Observer-Based Inverse Reinforcement Learning](https://mlsanity.com/paper/2011.02057/) (2020)
- [Online inverse reinforcement learning with limited data](https://mlsanity.com/paper/2008.08972/) (2020)
- [Data-Driven Inverse Reinforcement Learning for Expert-Learner Zero-Sum Games](https://mlsanity.com/paper/2301.01997/) (2023)
- [Data-Driven Inverse Optimal Control for Continuous-Time Nonlinear Systems](https://mlsanity.com/paper/2503.09090/) (2025)
- [Recursive Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2504.13241/) (2025)

## Abstract (arXiv)

This paper addresses the problem of online inverse reinforcement learning for nonlinear systems with modeling uncertainties while in the presence of unknown disturbances. The developed approach observes state and input trajectories for an agent and identifies the unknown reward function online. Sub-optimality introduced in the observed trajectories by the unknown external disturbance is compensated for using a novel model-based inverse reinforcement learning approach. The observer estimates the external disturbances and uses the resulting estimates to learn the dynamic model of the demonstrator. The learned demonstrator model along with the observed suboptimal trajectories are used to implement inverse reinforcement learning. Theoretical guarantees are provided using Lyapunov theory and a simulation example is shown to demonstrate the effectiveness of the proposed technique.