mlsanityData to 7 Oct 2026

Mature GAIL: Imitation Learning for Low-level and High-dimensional Input using Global Encoder and Cost Transformation

arXiv 1909.03200 ↗First posted 7 Sep 2019cs.LG stat.MLcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Recently, GAIL framework and various variants have shown remarkable possibilities for solving practical MDP problems. However, detailed researches of low-level, and high-dimensional state input in this framework, such as image sequences, has not been conducted. Furthermore, the cost function learned in the traditional GAIL frame-work only lies on a negative range, acting as a non-penalized reward and making the agent difficult to learn the optimal policy. In this paper, we propose a new algorithm based on the GAIL framework that includes a global encoder and the reward penalization mechanism. The global encoder solves two issues that arise when applying GAIL framework to high-dimensional image state. Also, it is shown that the penalization mechanism provides more adequate reward to the agent, resulting in stable performance improvement. Our approach's potential can be backed up by the fact that it is generally applicable to variants of GAIL framework. We conducted in-depth experiments by applying our methods to various variants of the GAIL framework. And, the results proved that our method significantly improves the performances when it comes to low-level and high-dimensional tasks.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 10 of its 19 references that are in the map. First posted on arXiv 7 Sep 2019.

Builds on

10 of 19 references.

Closest ancestors · 2

Same direction, newest first

  1. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  2. 2018Directed-Info GAIL: Learning Hierarchical Policies from Unsegmented Demonstrations using Directed Information29 Sep

Other references · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2015Trust Region Policy Optimizationcited by 2,503
  3. 2016OpenAI Gymcited by 1,684
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2018World Modelscited by 1,444
  6. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  7. 2017Deep Reinforcement Learning framework for Autonomous Drivingcited by 224
  8. 2016Control of Memory, Active Perception, and Action in Minecraftcited by 61

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Augmenting GAIL with BC for sample efficient imitation learning
  2. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  3. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence
  4. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate
  5. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
  6. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  7. 2019Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning
  8. 2020Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets