# Online inverse reinforcement learning with limited data

arXiv: https://arxiv.org/abs/2008.08972 · HTML: https://mlsanity.com/paper/2008.08972/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 5 of its 22 references that are in the map.
- First posted on arXiv 18 Aug 2020.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 5 of 5)

- [Online inverse reinforcement learning with unknown disturbances](https://mlsanity.com/paper/2003.03912/) (2020)
- [Inverse reinforcement learning in continuous time and space](https://mlsanity.com/paper/1801.07663/) (2018)
- [Online Output-Feedback Parameter and State Estimation for Second Order Linear Systems](https://mlsanity.com/paper/1609.05879/) (2016)
- [Approximate Optimal Trajectory Tracking for Continuous Time Nonlinear Systems](https://mlsanity.com/paper/1301.7664/) (2013)
- [Model-based reinforcement learning for infinite-horizon approximate optimal tracking](https://mlsanity.com/paper/1506.00685/) (2015)

## Built on by (most cited first, 2 of 2)

- [Online Observer-Based Inverse Reinforcement Learning](https://mlsanity.com/paper/2011.02057/) (2020)
- [Recursive Deep Inverse Reinforcement Learning](https://mlsanity.com/paper/2504.13241/) (2025)

## Abstract (arXiv)

This paper addresses the problem of online inverse reinforcement learning for systems with limited data and uncertain dynamics. In the developed approach, the state and control trajectories are recorded online by observing an agent perform a task, and reward function estimation is performed in real-time using a novel inverse reinforcement learning approach. Parameter estimation is performed concurrently to help compensate for uncertainties in the agent's dynamics. Data insufficiency is resolved by developing a data-driven update law to estimate the optimal feedback controller. The estimated controller can then be queried to artificially create additional data to drive reward function estimation.