# Fast Rates for Inverse Reinforcement Learning

arXiv: https://arxiv.org/abs/2605.14599 · HTML: https://mlsanity.com/paper/2605.14599/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 19 of its 43 references that are in the map.
- First posted on arXiv 14 May 2026.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)
- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 19 of 19)

- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [A Theory of Regularized Markov Decision Processes](https://mlsanity.com/paper/1901.11275/) (2019)
- [On the Global Convergence Rates of Softmax Policy Gradient Methods](https://mlsanity.com/paper/2005.06392/) (2020)
- [The Statistical Complexity of Interactive Decision Making](https://mlsanity.com/paper/2112.13487/) (2021)
- [IQ-Learn: Inverse soft-Q Learning for Imitation](https://mlsanity.com/paper/2106.12142/) (2021)
- [Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap](https://mlsanity.com/paper/2103.03236/) (2021)
- [Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning](https://mlsanity.com/paper/2407.15007/) (2024)
- [Identifiability in inverse reinforcement learning](https://mlsanity.com/paper/2106.03498/) (2021)
- [Finite-sample analysis of M-estimators using self-concordance](https://mlsanity.com/paper/1810.06838/) (2018)
- [Online Apprenticeship Learning](https://mlsanity.com/paper/2102.06924/) (2021)
- [When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning](https://mlsanity.com/paper/2302.07457/) (2023)
- [The Pitfalls of Imitation Learning when Actions are Continuous](https://mlsanity.com/paper/2503.09722/) (2025)
- [Identifiability and Generalizability in Constrained Inverse Reinforcement Learning](https://mlsanity.com/paper/2306.00629/) (2023)
- [Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecification](https://mlsanity.com/paper/2502.12465/) (2025)
- [Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learning](https://mlsanity.com/paper/2502.13900/) (2025)
- [Skill or Luck? Return Decomposition via Advantage Functions](https://mlsanity.com/paper/2402.12874/) (2024)
- [Towards the Transferability of Rewards Recovered via Regularized Inverse Reinforcement Learning](https://mlsanity.com/paper/2406.01793/) (2024)
- [Confidence Sets under Generalized Self-Concordance](https://mlsanity.com/paper/2301.00260/) (2022)
- [A Test-Function Approach to Incremental Stability](https://mlsanity.com/paper/2507.00695/) (2025)

## Built on by (most cited first, 1 of 1)

- [Provable Benefits of Regularization: Fast Rates for Adversarial Imitation Learning](https://mlsanity.com/paper/2609.35698/) (2026)

## Abstract (arXiv)

We establish novel structural and statistical results for entropy-regularized min-max inverse reinforcement learning (Min-Max-IRL) in finite-horizon MDPs with Borel state and action spaces. We show that maximum likelihood estimation (MLE) and Min-Max-IRL are equivalent at the population level, and at the empirical level under deterministic dynamics. For linear reward classes, we leverage pseudo-self-concordance of the Min-Max-IRL loss to prove that both the excess trajectory-level KL divergence and the squared parameter error in the Hessian norm decay at the fast rate O(n-1), where n is the number of expert trajectories. A local minimax lower bound matches the parameter-error rate up to logarithmic factors in the well-specified deterministic setting. Our guarantees apply under misspecification and require no uniform state-coverage assumption. We further extend reward-identifiability results to general Borel spaces and compare our results with MLE-based guarantees.