mlsanityData to 7 Oct 2026

Augmenting GAIL with BC for sample efficient imitation learning

arXiv 2001.07798 ↗First posted 21 Jan 2020cs.LG stat.MLcited by 4

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning is the problem of recovering an expert policy without access to a reward signal. Behavior cloning and GAIL are two widely used methods for performing imitation learning. Behavior cloning converges in a few iterations but doesn't achieve peak performance due to its inherent iid assumption about the state-action distribution. GAIL addresses the issue by accounting for the temporal dependencies when performing a state distribution matching between the agent and the expert. Although GAIL is sample efficient in the number of expert trajectories required, it is still not very sample efficient in terms of the environment interactions needed for convergence of the policy. Given the complementary benefits of both methods, we present a simple and elegant method to combine both methods to enable stable and sample efficient learning. Our algorithm is very simple to implement and integrates with different policy gradient algorithms. We demonstrate the effectiveness of the algorithm in low dimensional control tasks, gridworlds and in high dimensional image-based tasks.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 11 of its 32 references that are in the map. First posted on arXiv 21 Jan 2020.

Builds on

11 of 32 references.

References · 11

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  3. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  4. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  5. 2016Generative Adversarial Imitation Learningcited by 1,576
  6. 2017Wasserstein GANcited by 635
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2016Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimizationcited by 400
  9. 2018Rethinking ImageNet Pre-trainingcited by 376
  10. 2018Learning Goal Embeddings via Self-Play for Hierarchical Reinforcement Learningcited by 23
  11. 2019On Warm-Starting Neural Network Trainingcited by 9

Built on by

Cited by year

Citing papers per year: 2020: 1; 2021: 1; 2022: 1; 2023: 0; 2024: 1.

4 papers.

  1. 2022Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrationscited by 30
  2. 2021ReAgent: Point Cloud Registration using Imitation and Reinforcement Learningcited by 23
  3. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learningcited by 20
  4. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6

Similar, not cited

Close in wording, no citation either way

  1. 2021A Pragmatic Look at Deep Imitation Learning
  2. 2022Planning for Sample Efficient Imitation Learning
  3. 2019Imitation Learning as f-Divergence Minimization
  4. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  5. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  6. 2019Mature GAIL: Imitation Learning for Low-level and High-dimensional Input using Global Encoder and Cost Transformation
  7. 2022Diverse Imitation Learning via Self-Organizing Generative Models
  8. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory