mlsanityData to 13 Oct 2026

Inversely Learning Transferable Rewards via Abstracted States

arXiv 2501.01669 ↗First posted 3 Jan 2025cs.LG cs.ROcited by 1

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Inverse reinforcement learning (IRL) has progressed significantly toward accurately learning the underlying rewards in both discrete and continuous domains from behavior data. The next advance is to learn em intrinsic preferences in ways that produce useful behavior in settings or tasks which are different but aligned with the observed ones. In the context of robotic applications, this helps integrate robots into processing lines involving new tasks (with shared intrinsic preferences) without programming from scratch. We introduce a method to inversely learn an abstract reward function from behavior trajectories in two or more differing instances of a domain. The abstract reward function is then used to learn task behavior in another separate instance of the domain. This step offers evidence of its transferability and validates its correctness. We evaluate the method on trajectories in tasks from multiple domains in OpenAI's Gym testbed and AssistiveGym and show that the learned abstract reward functions can successfully learn task behaviors in instances of the respective domains, which have not been seen previously.

Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 47 references that are in the map. First posted on arXiv 3 Jan 2025.

Builds on

28 of 47 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Non-Adversarial Inverse Reinforcement Learning via Successor Feature Matching11 Nov
  2. 2024Towards the Transferability of Rewards Recovered via Regularized Inverse Reinforcement Learning3 Jun
  3. 2022Environment Design for Inverse Reinforcement Learning26 Oct
  4. 2022Identifiability and generalizability from multiple experts in Inverse Reinforcement Learning22 Sep
  5. 2022Lifelong Inverse Reinforcement Learning1 Jul
  6. 2022Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learning19 Jun
  7. 2021Identifiability in inverse reinforcement learning7 Jun
  8. 2020f-IRL: Inverse Reinforcement Learning via State Marginal Matching9 Nov
  9. 2020Robust Inverse Reinforcement Learning under Transition Dynamics Mismatch2 Jul
  10. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  11. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progress18 Jun

Other references · 17

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,180
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,656
  3. 2013Auto-Encoding Variational Bayescited by 4,024
  4. 2017Improved Training of Wasserstein GANscited by 2,226
  5. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,940
  6. 2016Generative Adversarial Imitation Learningcited by 1,577
  7. 2019A Comprehensive Survey on Transfer Learningcited by 1,157
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2021IQ-Learn: Inverse soft-Q Learning for Imitationcited by 191
  10. 2019A Divergence Minimization Perspective on Imitation Learning Methodscited by 188
  11. 2019Efficient Exploration via State Marginal Matchingcited by 186
  12. 2021XIRL: Cross-embodiment Inverse Reinforcement Learningcited by 102
5 more
  1. 2019Assistive Gym: A Physics Simulation Framework for Assistive Roboticscited by 44
  2. 2021Learning Markov State Abstractions for Deep Reinforcement Learningcited by 42
  3. 2020State-only Imitation with Transition Dynamics Mismatchcited by 37
  4. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  5. 2016Inverse Reinforcement Learning with Simultaneous Estimation of Rewards and Dynamicscited by 24

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026ConTraIRL: Factorized Contrastive Abstractions for Transferable IRLcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  2. 2021Versatile Inverse Reinforcement Learning via Cumulative Rewards
  3. 2022Basis for Intentions: Efficient Inverse Reinforcement Learning using Past Experience
  4. 2024Multi Task Inverse Reinforcement Learning for Common Sense Reward
  5. 2020Active Task-Inference-Guided Deep Inverse Reinforcement Learning
  6. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning
  7. 2022Inducing Structure in Reward Learning by Learning Features
  8. 2023Inverse Preference Learning: Preference-based RL without a Reward Function