mlsanityData to 14 Oct 2026

Inverse Decision Modeling: Learning Interpretable Representations of Behavior

arXiv 2310.18591 ↗First posted 28 Oct 2023stat.ML cs.LGcited by 10

Not on the map. Why?

Abstract (arXiv)

Decision analysis deals with modeling and enhancing decision processes. A principal challenge in improving behavior is in obtaining a transparent description of existing behavior in the first place. In this paper, we develop an expressive, unifying perspective on inverse decision modeling: a framework for learning parameterized representations of sequential decision behavior. First, we formalize the forward problem (as a normative standard), subsuming common classes of control behavior. Second, we use this to formalize the inverse problem (as a descriptive model), generalizing existing work on imitation/reward learning -- while opening up a much broader class of research problems in behavior representation. Finally, we instantiate this approach with an example (inverse bounded rational control), illustrating how this structure enables learning (interpretable) representations of (bounded) rationality -- while naturally capturing intuitive notions of suboptimal actions, biased beliefs, and imperfect knowledge of environments.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 68 of its 181 references that are in the map. First posted on arXiv 28 Oct 2023.

Builds on

68 of 181 references.

References · 68

  1. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  6. 2018An Algorithmic Perspective on Imitation Learningcited by 459
  7. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 406
  8. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  9. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  10. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 190
  11. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 183
  12. 2018Verifiable Reinforcement Learning via Policy Extractioncited by 177
38 more
  1. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
  2. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 168
  3. 2018Deep Variational Reinforcement Learning for POMDPscited by 125
  4. 2019Imitation Learning as f-Divergence Minimizationcited by 123
  5. 2020Primal Wasserstein Imitation Learningcited by 121
  6. 2020Reward-rational (implicit) choice: A unifying formalism for reward learningcited by 120
  7. 2014Generalization and Exploration via Randomized Value Functionscited by 112
  8. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferencescited by 73
  9. 2017Boltzmann Exploration Done Rightcited by 69
  10. 2020Online Bayesian Goal Inference for Boundedly-Rational Planning Agentscited by 68
  11. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matchingcited by 67
  12. 2020Quantifying Differences in Reward Functionscited by 63
  13. 2019Wasserstein Adversarial Imitation Learningcited by 60
  14. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  15. 2019Information asymmetry in KL-regularized RLcited by 53
  16. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  17. 2018On Improving Deep Reinforcement Learning for POMDPscited by 50
  18. 2019On the Feasibility of Learning, Rather than Assuming, Human Biases for Reward Inferencecited by 49
  19. 2019Variational Recurrent Models for Solving Partially Observable Control Taskscited by 47
  20. 2017Occam's razor is insufficient to infer the preferences of irrational agentscited by 45
  21. 2018Where Do You Think You're Going?: Inferring Beliefs about Dynamics from Behaviorcited by 40
  22. 2020Understanding Learned Reward Functionscited by 37
  23. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  24. 2020POPCORN: Partially Observed Prediction COnstrained ReiNforcement Learningcited by 36
  25. 2019Learning Causal State Representations of Partially Observable Environmentscited by 30
  26. 2016Model-based Adversarial Imitation Learningcited by 29
  27. 2017Repeated Inverse Reinforcement Learningcited by 29
  28. 2019Deep Bayesian Reward Learning from Preferencescited by 28
  29. 2019Truly Batch Apprenticeship Learning with Deep Successor Featurescited by 27
  30. 2019If MaxEnt RL is the Answer, What is the Question?cited by 27
  31. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24
  32. 2016Combining policy gradient and Q-learningcited by 24
  33. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimizationcited by 22
  34. 2015Information-Theoretic Bounded Rationalitycited by 21
  35. 2019Soft Policy Gradient Method for Maximum Entropy Deep Reinforcement Learningcited by 20
  36. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learningcited by 20
  37. 2020What Did You Think Would Happen? Explaining Agent Behaviour Through Intended Outcomescited by 20
  38. 2016Towards Resolving Unidentifiability in Inverse Reinforcement Learningcited by 18

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 4; 2025: 2; 2026: 2.

10 papers.

  1. 2024Beyond Preferences in AI Alignmentcited by 49
  2. 2023Time-series Generation by Contrastive Imitationcited by 9
  3. 2025OpenReview Should be Protected and Leveraged as a Community Asset for Research in the Era of Large Language Modelscited by 5
  4. 2024Reinforcement Learning Interventions on Boundedly Rational Human Agents in Frictionful Taskscited by 2
  5. 2024Defining Expertise: Applications to Treatment Effect Estimationcited by 2
  6. 2024Inverse Concave-Utility Reinforcement Learning is Inverse Game Theorycited by 2
  7. 2025More Than Irrational: Modeling Belief-Biased Agentscited by 2
  8. 2023Optimising Human-AI Collaboration by Learning Convincing Explanationscited by 1
  9. 2026Quantifying Potential Observation Missingness in Inverse Reinforcement Learningcited by 0
  10. 2026Reading Cognition as Decisions Unfold in Words: A Factorized Inverse Decision Modelcited by 0