mlsanityData to 14 Oct 2026

Generalize and Guide: Decomposing Rewards for Few-Shot Inverse Reinforcement Learning

arXiv 2607.17760 ↗First posted 20 Jul 2026cs.LG cs.AI cs.ROcited by 0

On the map: Offline RL and imitation learning from data → Inverse reinforcement learning

also in 1 more
Abstract (arXiv)

Inverse reinforcement learning (IRL) provides a powerful framework for learning from demonstrations. However, real-world tasks often exhibit substantial natural variations (e.g., picking up mugs with varying shapes), making it impractical to collect demonstrations that fully specify a new task under every possible scenario. In practice, while demonstrations for the target task are limited, it is often easier to obtain datasets of heterogeneous but related behaviors. This motivates the problem of few-shot IRL with multi-task demonstrations (FM-IRL), where an agent must learn a new task with substantial variations from only a limited number of target-task demonstrations, together with sufficient demonstrations of related tasks and online agent experience. To do so, we must both recover the expert distribution of the new task and provide guidance when the agent deviates from it. We introduce Multitask discriminator Proximity-Guided IRL (MPG), which learns two complementary reward components: (1) a generalizable discriminator that transfers shared structure across related tasks to identify expert behavior in a new task, and (2) a proximity function that measures how far a state deviates from expert behavior and provides corrective guidance during exploration. We demonstrate the effectiveness of our method on multiple challenging navigation and manipulation tasks under significant variations (e.g., object configurations, table layouts, and initial robot poses), achieving an average success rate of 81.2%, outperforming the strongest per-task baseline by an average of 24.7 percentage points.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Inverse reinforcement learning, one of 13 directions in Offline RL and imitation learning from data. Builds on 28 of its 37 references that are in the map. First posted on arXiv 20 Jul 2026.

Builds on

28 of 37 references.

Closest ancestors · 4

Same direction, newest first

  1. 2023Multi-task Hierarchical Adversarial Inverse Reinforcement Learning22 May
  2. 2019Meta-Inverse Reinforcement Learning with Probabilistic Context Variables20 Sep
  3. 2018Learning a Prior over Intent via Meta-Inverse Reinforcement Learning31 May
  4. 2018Multi-task Maximum Entropy Inverse Reinforcement Learning22 May

Other references · 24

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  3. 2016Concrete Problems in AI Safetycited by 1,642
  4. 2016Generative Adversarial Imitation Learningcited by 1,597
  5. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  7. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  8. 2023Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Taskscited by 327
  9. 2017One-Shot Imitation Learningcited by 233
  10. 2017One-Shot Visual Imitation Learning via Meta-Learningcited by 228
  11. 2020Accelerating Reinforcement Learning with Learned Skill Priorscited by 223
  12. 2018One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learningcited by 148
12 more
  1. 2020Primal Wasserstein Imitation Learningcited by 121
  2. 2023Optimal Goal-Reaching Reinforcement Learning via Quasimetric Learningcited by 121
  3. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 119
  4. 2022Discriminator-Weighted Offline Imitation Learning from Suboptimal Demonstrationscited by 89
  5. 2023Decomposing the Generalization Gap in Imitation Learning for Visual Robotic Manipulationcited by 82
  6. 2022Watch and Match: Supercharging Imitation with Regularized Optimal Transportcited by 67
  7. 2023Optimal Transport for Offline Imitation Learningcited by 42
  8. 2021Hierarchical Few-Shot Imitation with Skill Transition Modelscited by 37
  9. 2018Few-Shot Goal Inference for Visuomotor Learning and Planningcited by 30
  10. 2022Robust Imitation Learning against Variations in Environment Dynamicscited by 26
  11. 2021PsiPhi-Learning: Reinforcement Learning with Demonstrations using Successor Features and Inverse Temporal Difference Learningcited by 23
  12. 2024Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learningcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrations
  2. 2024Multi Task Inverse Reinforcement Learning for Common Sense Reward
  3. 2025Generalizing Behavior via Inverse Reinforcement Learning with Closed-Form Reward Centroids
  4. 2020Inverse Reinforcement Learning via Matching of Optimality Profiles
  5. 2026Scalable Multi-Task Inverse Reinforcement Learning
  6. 2023Distance-rank Aware Sequential Reward Learning for Inverse Reinforcement Learning with Sub-optimal Demonstrations
  7. 2022Context-Hierarchy Inverse Reinforcement Learning
  8. 2025Reward Compatibility: A Framework for Inverse RL