Fast Rates for Inverse Reinforcement Learning
On the map: Offline RL and imitation learning from data → Inverse reinforcement learning
Abstract (arXiv)
We establish novel structural and statistical results for entropy-regularized min-max inverse reinforcement learning (Min-Max-IRL) in finite-horizon MDPs with Borel state and action spaces. We show that maximum likelihood estimation (MLE) and Min-Max-IRL are equivalent at the population level, and at the empirical level under deterministic dynamics. For linear reward classes, we leverage pseudo-self-concordance of the Min-Max-IRL loss to prove that both the excess trajectory-level KL divergence and the squared parameter error in the Hessian norm decay at the fast rate O(n-1), where n is the number of expert trajectories. A local minimax lower bound matches the parameter-error rate up to logarithmic factors in the well-specified deterministic setting. Our guarantees apply under misspecification and require no uniform state-coverage assumption. We further extend reward-identifiability results to general Borel spaces and compare our results with MLE-based guarantees.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 19 of its 43 references that are in the map. First posted on arXiv 14 May 2026.
Builds on
19 of 43 references.
Closest ancestors · 4
Same direction, newest first
- 2024Towards the Transferability of Rewards Recovered via Regularized Inverse Reinforcement Learning3 Jun
- 2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learning1 Jun
- 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learning15 Feb
- 2021Identifiability in inverse reinforcement learning7 Jun
Other references · 15
- 2016Generative Adversarial Imitation Learningcited by 1,597
- 2019A Theory of Regularized Markov Decision Processescited by 286
- 2020On the Global Convergence Rates of Softmax Policy Gradient Methodscited by 264
- 2021The Statistical Complexity of Interactive Decision Makingcited by 218
- 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
- 2021Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gapcited by 81
- 2024Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learningcited by 74
- 2018Finite-sample analysis of M-estimators using self-concordancecited by 30
- 2021Online Apprenticeship Learningcited by 29
- 2025The Pitfalls of Imitation Learning when Actions are Continuouscited by 26
- 2025Computational-Statistical Tradeoffs at the Next-Token Prediction Barrier: Autoregressive and Imitation Learning under Misspecificationcited by 13
- 2025Optimistically Optimistic Exploration for Provably Efficient Infinite-Horizon Reinforcement and Imitation Learningcited by 12
3 more
- 2024Skill or Luck? Return Decomposition via Advantage Functionscited by 8
- 2022Confidence Sets under Generalized Self-Concordancecited by 1
- 2025A Test-Function Approach to Incremental Stabilitycited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2020Revisiting Maximum Entropy Inverse Reinforcement Learning: New Perspectives and Algorithms
- 2019Generalized Maximum Causal Entropy for Inverse Reinforcement Learning
- 2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learning
- 2024Convergence of a model-free entropy-regularized inverse reinforcement learning algorithm
- 2021A Generalised Inverse Reinforcement Learning Framework
- 2026Maximum Entropy Inverse Reinforcement Learning for Mean-Field Games with Average Reward
- 2023A Novel Variational Lower Bound for Inverse Reinforcement Learning
- 2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guarantees