Data to 7 Oct 2026

Identifiability in inverse reinforcement learning

arXiv 2106.03498 ↗First posted 7 Jun 2021cs.LG math.OCcited by 55

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning attempts to reconstruct the reward function in a Markov decision problem, using observations of agent actions. As already observed in Russell [1998] the problem is ill-posed, and the reward function is not identifiable, even under the presence of perfect information about optimal behavior. We provide a resolution to this non-identifiability for problems with entropy regularization. For a given environment, we fully characterize the reward functions leading to a given policy and demonstrate that, given demonstrations of actions for the same reward under two distinct discount factors, or under sufficiently different environments, the unobserved reward can be recovered up to a constant. We also give general necessary and sufficient conditions for reconstruction of time-homogeneous rewards on finite horizons, and for action-independent rewards, generalizing recent results of Kim et al. [2021] and Fu et al. [2018].

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 44 references that are in the map. First posted on arXiv 7 Jun 2021.

Builds on

8 of 44 references.

Closest ancestors · 1

Same direction, newest first

2020Efficient Exploration of Reward Functions in Inverse Reinforcement Learning via Bayesian Optimization17 Nov

Other references · 7

2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
2018Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Reviewcited by 466
2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
2017Repeated Inverse Reinforcement Learningcited by 29
2016Towards Resolving Unidentifiability in Inverse Reinforcement Learningcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 9; 2024: 13; 2025: 13; 2026: 10.

55 papers.

Sort:

By area:

2024From r to Q^*: Your Language Model is Secretly a Q-Functioncited by 242
2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
2022Maximum-Likelihood Inverse Reinforcement Learning with Finite-Time Guaranteescited by 42
2022Misspecification in Inverse Reinforcement Learningcited by 27
2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
2022On the Sensitivity of Reward Inference to Misspecified Human Modelscited by 25
2023Coherent Soft Imitation Learningcited by 21
2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learningcited by 16
2023Identifiability and Generalizability in Constrained Inverse Reinforcement Learningcited by 16
2022A Primer on Maximum Causal Entropy Inverse Reinforcement Learningcited by 13
2022Utilizing Prior Solutions for Reward Shaping and Composition in Entropy-Regularized Reinforcement Learningcited by 8

38 more

2023K-SHAP: Policy Clustering Algorithm for Anonymous Multi-Agent State-Action Pairscited by 6
2024Towards the Transferability of Rewards Recovered via Regularized Inverse Reinforcement Learningcited by 6
2022Transferable Reward Learning by Dynamics-Agnostic Discriminator Ensemblecited by 5
2024How does Inverse RL Scale to Large State Spaces? A Provably Efficient Approachcited by 5
2025Agency Is Frame-Dependentcited by 5
2022Calculus on MDPs: Potential Shaping as a Gradientcited by 4
2022Environment Design for Inverse Reinforcement Learningcited by 4
2023FoMo Rewards: Can we cast foundation models as reward functions?cited by 4
2024Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecificationcited by 4
2024Boosting Soft Q-Learning by Boundingcited by 4
2024On Feasible Rewards in Multi-Agent Inverse Reinforcement Learningcited by 4
2025Inverse Reinforcement Learning with Just Classification and a Few Regressionscited by 4
2024Rethinking Inverse Reinforcement Learning: from Data Alignment to Task Alignmentcited by 3
2025Bootstrapped Reward Shapingcited by 3
2025Learning Reward Machines from Partially Observed Policiescited by 3
2023Eliciting Risk Aversion with Inverse Reinforcement Learning via Interactive Questioningcited by 2
2023Toward Computationally Efficient Inverse Reinforcement Learning via Reward Shapingcited by 2
2025Inversely Learning Transferable Rewards via Abstracted Statescited by 1
2025Model-Based Soft Maximization of Suitable Metrics of Long-Term Human Powercited by 1
2025Efficient Reward Identification In Max Entropy Reinforcement Learning with Sparsity and Rank Priorscited by 1
2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroidscited by 1
2026Fast Rates for Inverse Reinforcement Learningcited by 1
2026Inverting the Bellman Equation: From Q-Values to World Modelscited by 1
2023Leveraging Prior Knowledge in Reinforcement Learning via Double-Sided Bounds on the Value Functioncited by 0
2023PAGAR: Taming Reward Misalignment in Inverse Reinforcement Learning-Based Imitation Learning with Protagonist Antagonist Guided Adversarial Rewardcited by 0
2024Rethinking Adversarial Inverse Reinforcement Learning: Policy Imitation, Transferable Reward Recovery and Algebraic Equilibrium Proofcited by 0
2024Randomized algorithms and PAC bounds for inverse reinforcement learning in continuous spacescited by 0
2024Inference of Utilities and Time Preference in Sequential Decision-Makingcited by 0
2024Inverse Reinforcement Learning with Multiple Planning Horizonscited by 0
2024On Reward Transferability in Adversarial Inverse Reinforcement Learning: Insights from Random Matrix Theorycited by 0
2025Robustness in the Face of Partial Identifiability in Reward Learningcited by 0
2025Where You Go is Who You Are: Behavioral Theory-Guided LLMs for Inverse Reinforcement Learningcited by 0
2025Icon2: Aligning Large Language Models Using Self-Synthetic Preference Data via Inherent Regulationcited by 0
2025Blind Inverse Game Theory: Jointly Decoding Rewards and Rationality in Entropy-Regularized Competitive Gamescited by 0
2025Statistical analysis of Inverse Entropy-regularized Reinforcement Learningcited by 0
2026Notes on the Reward Representation of Posterior Updatescited by 0
2026Active Reward Machine Inference From Raw State Trajectoriescited by 0
2026Learning the Preferences of a Learning Agentcited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way