mlsanityData to 7 Oct 2026

A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress

arXiv 1806.06877 ↗First posted 18 Jun 2018cs.LG stat.MLcited by 294

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

Inverse reinforcement learning (IRL) is the problem of inferring the reward function of an agent, given its policy or observed behavior. Analogous to RL, IRL is perceived both as a problem and as a class of methods. By categorically surveying the current literature in IRL, this article serves as a reference for researchers and practitioners of machine learning and beyond to understand the challenges of IRL and select the approaches best suited for the problem on hand. The survey formally introduces the IRL problem along with its central challenges such as the difficulty in performing accurate inference and its generalizability, its sensitivity to prior knowledge, and the disproportionate growth in solution complexity with problem size. The article elaborates how the current methods mitigate these challenges. We further discuss the extensions to traditional IRL methods for handling: inaccurate and incomplete perception, an incomplete model, multiple reward functions, and nonlinear reward functions. This survey concludes the discussion with some broad advances in the research area and currently open research questions.

Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 118 references that are in the map. First posted on arXiv 18 Jun 2018.

Builds on

8 of 118 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Maximum Causal Tsallis Entropy Imitation Learning22 May

Other references · 7

  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  3. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  4. 2013The Complexity of Decentralized Control of Markov Decision Processescited by 225
  5. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 173
  6. 2013Computational Rationalization: The Inverse Equilibrium Problemcited by 24
  7. 2017Efficient Probabilistic Performance Bounds for Inverse Reinforcement Learningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 5; 2020: 5; 2021: 28; 2022: 33; 2023: 64; 2024: 74; 2025: 48; 2026: 37.

294 papers.

Sort:

By area:

  1. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 256
  2. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  3. 2026Self-Distillation Enables Continual Learningcited by 212
  4. 2021A Survey of Zero-shot Generalisation in Deep Reinforcement Learningcited by 164
  5. 2024AI Agents Under Threat: A Survey of Key Security Challenges and Future Pathwayscited by 122
  6. 2021ThriftyDAgger: Budget-Aware Novelty and Risk Gating for Interactive Imitation Learningcited by 98
  7. 2024RLHF Deciphered: A Critical Analysis of Reinforcement Learning from Human Feedback for LLMscited by 89
  8. 2021A Survey on Interpretable Reinforcement Learningcited by 70
  9. 2021Imitation Learning: Progress, Taxonomies and Challengescited by 56
  10. 2023Benchmarks and Algorithms for Offline Preference-Based Reward Learningcited by 55
  11. 2022Models of human preference for learning reward functionscited by 54
  12. 2021LazyDAgger: Reducing Context Switching in Interactive Imitation Learningcited by 50
38 more
  1. 2023DTPP: Differentiable Joint Conditional Prediction and Cost Evaluation for Tree Policy Planning in Autonomous Drivingcited by 46
  2. 2023Language Reward Modulation for Pretraining Reinforcement Learningcited by 42
  3. 2023PARL: A Unified Framework for Policy Alignment in Reinforcement Learning from Human Feedbackcited by 38
  4. 2021Reinforcement Learning for Ridesharing: An Extended Surveycited by 35
  5. 2024Comprehensive Overview of Reward Engineering and Shaping in Advancing Reinforcement Learning Applicationscited by 34
  6. 2021INVIGORATE: Interactive Visual Grounding and Grasping in Cluttercited by 32
  7. 2022Trustworthy Reinforcement Learning Against Intrinsic Vulnerabilities: Robustness, Safety, and Generalizabilitycited by 32
  8. 2019Driving with Style: Inverse Reinforcement Learning in General-Purpose Planning for Automated Drivingcited by 30
  9. 2023Augmenting Autotelic Agents with Large Language Modelscited by 29
  10. 2023Towards Knowledge-driven Autonomous Drivingcited by 29
  11. 2023Promptable Behaviors: Personalizing Multi-Objective Rewards from Human Preferencescited by 25
  12. 2019Federated Imitation Learning: A Novel Framework for Cloud Robotic Systems with Heterogeneous Sensor Datacited by 24
  13. 2024SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learningcited by 24
  14. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  15. 2021Cognitive science as a source of forward and inverse models of human decisions for robotics and controlcited by 22
  16. 2023Accelerating Reinforcement Learning of Robotic Manipulations via Feedback from Large Language Modelscited by 22
  17. 2022Your Policy Regularizer is Secretly an Adversarycited by 21
  18. 2023Towards Theoretical Understanding of Inverse Reinforcement Learningcited by 21
  19. 2024Highly Efficient Self-Adaptive Reward Shaping for Reinforcement Learningcited by 21
  20. 2024From Novice to Expert: LLM Agent Policy Optimization via Step-wise Reinforcement Learningcited by 21
  21. 2021Contextual Inverse Optimization: Offline and Online Learningcited by 20
  22. 2022Language Control Diffusion: Efficiently Scaling through Space, Time, and Taskscited by 20
  23. 2023CLARE: Conservative Model-Based Reward Learning for Offline Inverse Reinforcement Learningcited by 20
  24. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  25. 2021Offline Preference-Based Apprenticeship Learningcited by 17
  26. 2023Extracting Reward Functions from Diffusion Modelscited by 17
  27. 2023Visual Imitation Learning with Patch Rewardscited by 16
  28. 2024Adversarial Safety-Critical Scenario Generation using Naturalistic Human Driving Priorscited by 15
  29. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  30. 2024SEABO: A Simple Search-Based Method for Offline Imitation Learningcited by 14
  31. 2024Goals as Reward-Producing Programscited by 14
  32. 2021Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Taskscited by 13
  33. 2022Learning Soft Constraints From Constrained Expert Demonstrationscited by 13
  34. 2024Zero-shot Imitation Policy via Search in Demonstration Datasetcited by 13
  35. 2021Risk-averse autonomous systems: A brief history and recent developments from the perspective of optimal controlcited by 12
  36. 2023A Survey of Demonstration Learningcited by 12
  37. 2023Principal-Agent Reward Shaping in MDPscited by 12
  38. 2024Zeroth-Order Policy Gradient for Reinforcement Learning from Human Feedback without Reward Inferencecited by 12

50 most cited of 294 All 294 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective
  2. 2025Reward Compatibility: A Framework for Inverse RL
  3. 2024Offline Inverse RL: New Solution Concepts and Provably Efficient Algorithms
  4. 2026Inverse Reinforcement Learning without an Optimal Demonstrator: A Feasible Reward Set Approach
  5. 2022Active Exploration for Inverse Reinforcement Learning
  6. 2018A Framework and Method for Online Inverse Reinforcement Learning
  7. 2023Inverse Reinforcement Learning without Reinforcement Learning
  8. 2026A Lecture Note on Offline RL and IRL, Part II: Foundations of Inverse Reinforcement Learning and Dynamic Discrete Choice Models