# Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models

arXiv: https://arxiv.org/abs/2512.24407 · HTML: https://mlsanity.com/paper/2512.24407/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 12 of its 85 references that are in the map.
- First posted on arXiv 30 Dec 2025.

## Placed in

- Offline RL and imitation learning from data / [Inverse reinforcement learning](https://mlsanity.com/offlinerl/inverse-reinforcement-learning/)

## Builds on (most cited first, 12 of 12)

- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [GenDICE: Generalized Offline Estimation of Stationary Values](https://mlsanity.com/paper/2002.09072/) (2020)
- [Doubly Robust Bias Reduction in Infinite Horizon Off-Policy Estimation](https://mlsanity.com/paper/1910.07186/) (2019)
- [Choice modelling in the age of machine learning -- discussion paper](https://mlsanity.com/paper/2101.11948/) (2021)
- [Off-Policy Evaluation in Markov Decision Processes under Weak Distributional Overlap](https://mlsanity.com/paper/2402.08201/) (2024)
- [Semiparametric Double Reinforcement Learning with Applications to Long-Term Causal Inference](https://mlsanity.com/paper/2501.06926/) (2025)
- [Automatic Debiased Machine Learning for Smooth Functionals of Nonparametric M-Estimands](https://mlsanity.com/paper/2501.11868/) (2025)
- [Inverse Reinforcement Learning with Just Classification and a Few Regressions](https://mlsanity.com/paper/2509.21172/) (2025)
- [Partial Identifiability and Misspecification in Inverse Reinforcement Learning](https://mlsanity.com/paper/2411.15951/) (2024)
- [An Empirical Risk Minimization Approach for Offline Inverse RL and Dynamic Discrete Choice Model](https://mlsanity.com/paper/2502.14131/) (2025)
- [The Value of Personalized Recommendations: Evidence from Netflix](https://mlsanity.com/paper/2511.07280/) (2025)
- [Fitted Q-Evaluation without Bellman Completeness via Occupancy Weighting](https://mlsanity.com/paper/2512.23805/) (2025)

## Built on by (most cited first, 3 of 3)

- [Reward Transfer from Inverse Reinforcement Learning: A Coupled Minimax Approach](https://mlsanity.com/paper/2605.27834/) (2026)
- [Fitted Occupancy-Ratio Evaluation without Bellman Completeness](https://mlsanity.com/paper/2607.05375/) (2026)
- [Bellman Calibration for Marginalized Importance Weighting in Offline Reinforcement Learning](https://mlsanity.com/paper/2608.24858/) (2026)

## Abstract (arXiv)

In many sequential decision-making problems, researchers observe actions but not the rewards that drive behavior, yet still wish to evaluate and compare counterfactual policies. Inverse reinforcement learning (IRL) and dynamic discrete choice (DDC) models address this setting by positing an optimality model that links latent rewards to observed actions. Existing flexible IRL methods allow rich reward representations but typically do not provide valid inference, whereas classical DDC methods support inference only under restrictive parametric structure. We develop a semiparametric framework for debiased inverse reinforcement learning in maximum-entropy IRL and Gumbel-shock DDC models. Our key identification result is that the log-behavior policy can be treated as a pseudo-reward: it point-identifies policy value differences and, under a normalization constraint, the reward itself. This reduces inference on reward-dependent estimands to inference on smooth functionals of the behavior policy and transition kernel. We establish pathwise differentiability, derive efficient influence functions, and construct automatic debiased machine-learning estimators that permit flexible nuisance estimation while attaining √n-consistency, asymptotic normality, and semiparametric efficiency. The result is a computationally tractable framework for valid uncertainty quantification in flexible IRL and DDC models.