# Cost Function Estimation Using Inverse Reinforcement Learning with Minimal Observations

arXiv: https://arxiv.org/abs/2505.08619 · HTML: https://mlsanity.com/paper/2505.08619/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 2 of its 22 references that are in the map.
- First posted on arXiv 13 May 2025.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 2 of 2)

- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control](https://mlsanity.com/paper/1909.04947/) (2019)

## Built on by (most cited first, 2 of 2)

- [Learning Human Reaching Optimality Principles from Minimal Observation Inverse Reinforcement Learning](https://mlsanity.com/paper/2510.00329/) (2025)
- [Toward Global Intent Inference for Human Motion by Inverse Reinforcement Learning](https://mlsanity.com/paper/2603.07797/) (2026)

## Abstract (arXiv)

We present an iterative inverse reinforcement learning algorithm to infer optimal cost functions in continuous spaces. Based on a popular maximum entropy criteria, our approach iteratively finds a weight improvement step and proposes a method to find an appropriate step size that ensures learned cost function features remain similar to the demonstrated trajectory features. In contrast to similar approaches, our algorithm can individually tune the effectiveness of each observation for the partition function and does not need a large sample set, enabling faster learning. We generate sample trajectories by solving an optimal control problem instead of random sampling, leading to more informative trajectories. The performance of our method is compared to two state of the art algorithms to demonstrate its benefits in several simulated environments.