mlsanityData to 14 Oct 2026

Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience

arXiv 2208.04919 ↗First posted 9 Aug 2022cs.LGcited by 4

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

Abstract (arXiv)

This paper addresses the problem of inverse reinforcement learning (IRL) -- inferring the reward function of an agent from observing its behavior. IRL can provide a generalizable and compact representation for apprenticeship learning, and enable accurately inferring the preferences of a human in order to assist them. %and provide for more accurate prediction. However, effective IRL is challenging, because many reward functions can be compatible with an observed behavior. We focus on how prior reinforcement learning (RL) experience can be leveraged to make learning these preferences faster and more efficient. We propose the IRL algorithm BASIS (Behavior Acquisition through Successor-feature Intention inference from Samples), which leverages multi-task RL pre-training and successor features to allow an agent to build a strong basis for intentions that spans the space of possible goals in a given domain. When exposed to just a few expert demonstrations optimizing a novel goal, the agent uses its basis to quickly and effectively infer the reward function. Our experiments reveal that our method is highly effective at inferring and optimizing demonstrated reward functions, accurately inferring reward functions from less than 100 trajectories.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 34 of its 56 references that are in the map. First posted on arXiv 9 Aug 2022.

Builds on

34 of 56 references.

Closest ancestors · 6

Same direction, newest first

  1. 2021Scalable Bayesian Inverse Reinforcement Learning12 Feb
  2. 2020Deep Inverse Q-learning with Constraints4 Aug
  3. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  4. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observations12 Apr
  5. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May
  6. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May

Other references · 28

  1. 2016Generative Adversarial Imitation Learningcited by 1,597
  2. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 669
  3. 2018Behavioral Cloning from Observationcited by 492
  4. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  5. 2017Time-Contrastive Networks: Self-Supervised Learning from Videocited by 459
  6. 2016Successor Features for Transfer in Reinforcement Learningcited by 340
  7. 2016Cooperative Inverse Reinforcement Learningcited by 300
  8. 2018Machine Theory of Mindcited by 226
  9. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learningcited by 204
  10. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 193
  11. 2019Imitation Learning via Off-Policy Distribution Matchingcited by 183
  12. 2015Maximum Entropy Deep Inverse Reinforcement Learningcited by 176
16 more
  1. 2017Imitation from Observation: Learning to Imitate Behaviors from Raw Video via Context Translationcited by 167
  2. 2019Fast Task Inference with Variational Intrinsic Successor Featurescited by 133
  3. 2018Count-Based Exploration with the Successor Representationcited by 112
  4. 2018Universal Successor Features Approximatorscited by 103
  5. 2017Eigenoption Discovery through the Deep Successor Representationcited by 86
  6. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  7. 2020Driving Behavior Modeling using Naturalistic Human Driving Data with Inverse Reinforcement Learningcited by 46
  8. 2017Visual Semantic Planning using Deep Successor Representationscited by 45
  9. 2018Successor Uncertainties: Exploration and Uncertainty in Temporal Difference Learningcited by 35
  10. 2016Model-based Adversarial Imitation Learningcited by 29
  11. 2019Truly Batch Apprenticeship Learning with Deep Successor Featurescited by 27
  12. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  13. 2017Observational Learning by Reinforcement Learningcited by 12
  14. 2017Advantages and Limitations of using Successor Features for Transfer in Reinforcement Learningcited by 10
  15. 2015Inverse Reinforcement Learning via Deep Gaussian Processcited by 6
  16. 2021Modeling human intention inference in continuous 3D domains by inverse planning and body kinematicscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 2; 2025: 0; 2026: 1.

4 papers.

  1. 2023When Demonstrations Meet Generative World Models: A Maximum Likelihood Framework for Offline Inverse Reinforcement Learningcited by 26
  2. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matchingcited by 10
  3. 2024OMG-RL:Offline Model-based Guided Reward Learning for Heparin Treatmentcited by 1
  4. 2026Do as the Romans Do: Learning Universal Behaviors from Heterogeneous Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress
  2. 2024Automated Feature Selection for Inverse Reinforcement Learning
  3. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  4. 2021Deep Adaptive Multi-Intention Inverse Reinforcement Learning
  5. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  6. 2020Inverse Reinforcement Learning from a Gradient-based Learner
  7. 2025Inversely Learning Transferable Rewards via Abstracted States
  8. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations