mlsanityData to 7 Oct 2026

Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning

arXiv 1911.00238 ↗First posted 1 Nov 2019cs.LG cs.AI cs.ROcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Generative adversarial imitation learning (GAIL) has attracted increasing attention in the field of robot learning. It enables robots to learn a policy to achieve a task demonstrated by an expert while simultaneously estimating the reward function behind the expert's behaviors. However, this framework is limited to learning a single task with a single reward function. This study proposes an extended framework called situated GAIL (S-GAIL), in which a task variable is introduced to both the discriminator and generator of the GAIL framework. The task variable has the roles of discriminating different contexts and making the framework learn different reward functions and policies for multiple tasks. To achieve the early convergence of learning and robustness during reward estimation, we introduce a term to adjust the entropy regularization coefficient in the generator's objective function. Our experiments using two setups (navigation in a discrete grid world and arm reaching in a continuous space) demonstrate that the proposed framework can acquire multiple reward functions and policies more effectively than existing frameworks. The task variable enables our framework to differentiate contexts while sharing common knowledge among multiple tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 16 of its 30 references that are in the map. First posted on arXiv 1 Nov 2019.

Builds on

16 of 30 references.

Closest ancestors · 1

Same direction, newest first

  1. 2018Global overview of Imitation Learning19 Jan

Other references · 15

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  6. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2019Learning Latent Plans from Playcited by 310
  9. 2017Bridging the Gap Between Value and Policy Based Reinforcement Learningcited by 178
  10. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  11. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  12. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
3 more
  1. 2016Connecting Generative Adversarial Networks and Actor-Critic Methodscited by 49
  2. 2017OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learningcited by 40
  3. 2019Improvisation through Physical Understanding: Using Novel Objects as Tools with Visual Foresightcited by 34

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 2; 2022: 0; 2023: 0; 2024: 1.

4 papers.

  1. 2021Offline Inverse Reinforcement Learningcited by 14
  2. 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4
  3. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  4. 2021A Generalised Inverse Reinforcement Learning Frameworkcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  2. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence
  3. 2020Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets
  4. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  5. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
  6. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  7. 2021A Pragmatic Look at Deep Imitation Learning
  8. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulation