mlsanityData to 7 Oct 2026

Multi-intention Inverse Q-learning for Interpretable Behavior Representation

arXiv 2311.13870 ↗First posted 23 Nov 2023cs.LG q-bio.NCcited by 9

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

In advancing the understanding of natural decision-making processes, inverse reinforcement learning (IRL) methods have proven instrumental in reconstructing animal's intentions underlying complex behaviors. Given the recent development of a continuous-time multi-intention IRL framework, there has been persistent inquiry into inferring discrete time-varying rewards with IRL. To address this challenge, we introduce the class of hierarchical inverse Q-learning (HIQL) algorithms. Through an unsupervised learning process, HIQL divides expert trajectories into multiple intention segments, and solves the IRL problem independently for each. Applying HIQL to simulated experiments and several real animal behavior datasets, our approach outperforms current benchmarks in behavior prediction and produces interpretable reward functions. Our results suggest that the intention transition dynamics underlying complex decision-making behavior is better modeled by a step function instead of a smoothly varying function. This advancement holds promise for neuroscience and cognitive science, contributing to a deeper understanding of decision-making and uncovering underlying brain mechanisms.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 8 of its 44 references that are in the map. First posted on arXiv 23 Nov 2023.

Builds on

8 of 44 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022Option-Aware Adversarial Inverse Reinforcement Learning for Robotic Control5 Oct
  2. 2021Deep Adaptive Multi-Intention Inverse Reinforcement Learning14 Jul
  3. 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
  4. 2020Deep Inverse Q-learning with Constraints4 Aug
  5. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun
  6. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May

Other references · 2

  1. 2022Graph Inverse Reinforcement Learning from Diverse Videoscited by 45
  2. 2019Inverse Rational Control with Partially Observable Continuous Nonlinear Dynamicscited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 7; 2026: 2.

9 papers.

  1. 2025Inverse Reinforcement Learning with Switching Rewards and History Dependency for Characterizing Animal Behaviorscited by 6
  2. 2025Learning Reward Machines from Partially Observed Policiescited by 3
  3. 2025Solving Inverse Problem for Multi-armed Bandits via Convex Optimizationcited by 2
  4. 2025Multi-convex Programming for Discrete Latent Factor Models Prototypingcited by 1
  5. 2025Learning Task-Agnostic Motifs to Capture the Continuous Nature of Animal Behaviorcited by 1
  6. 2025Inverse Reinforcement Learning via Convex Optimizationcited by 0
  7. 2025Fitting Reinforcement Learning Model to Behavioral Data under Banditscited by 0
  8. 2026Probabilistic Recurrent Intention Switching Modelcited by 0
  9. 2026Exploratory Experience Shapes the Geometry of Predictive Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026CoMI-IRL: Contrastive Multi-Intention Inverse Reinforcement Learning
  2. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
  3. 2022Context-Hierarchy Inverse Reinforcement Learning
  4. 2025BiCQL-ML: A Bi-Level Conservative Q-Learning Framework for Maximum Likelihood Inverse Reinforcement Learning
  5. 2025Inverse Reinforcement Learning with Just Classification and a Few Regressions
  6. 2018Inverse Reinforcement Learning via Nonparametric Spatio-Temporal Subgoal Modeling
  7. 2025Efficient Inference for Inverse Reinforcement Learning and Dynamic Discrete Choice Models
  8. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids