mlsanityData to 7 Oct 2026

Sample-Efficient Imitation Learning via Generative Adversarial Nets

arXiv 1809.02064 ↗First posted 6 Sep 2018cs.LG stat.MLcited by 36

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

GAIL is a recent successful imitation learning architecture that exploits the adversarial training procedure introduced in GANs. Albeit successful at generating behaviours similar to those demonstrated to the agent, GAIL suffers from a high sample complexity in the number of interactions it has to carry out in the environment in order to achieve satisfactory performance. We dramatically shrink the amount of interactions with the environment necessary to learn well-behaved imitation policies, by up to several orders of magnitude. Our framework, operating in the model-free regime, exhibits a significant increase in sample-efficiency over previous methods by simultaneously a) learning a self-tuned adversarially-trained surrogate reward and b) leveraging an off-policy actor-critic architecture. We show that our approach is simple to implement and that the learned agents remain remarkably stable, as shown in our experiments that span a variety of continuous control tasks. Video visualisations available at: urlhttps://youtu.be/-nCsqUJnRKU.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 34 of its 57 references that are in the map. First posted on arXiv 6 Sep 2018.

Builds on

34 of 57 references.

References · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2016Layer Normalizationcited by 5,579
  4. 2013Auto-Encoding Variational Bayescited by 3,980
  5. 2015Continuous control with deep reinforcement learningcited by 3,279
  6. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2015Trust Region Policy Optimizationcited by 2,503
  9. 2017Improved Training of Wasserstein GANscited by 2,223
  10. 2016OpenAI Gymcited by 1,684
  11. 2016Generative Adversarial Imitation Learningcited by 1,576
  12. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
22 more
  1. 2015Prioritized Experience Replaycited by 1,070
  2. 2017Rainbow: Combining Improvements in Deep Reinforcement Learningcited by 774
  3. 2017Deep Reinforcement Learning that Matterscited by 727
  4. 2017Wasserstein GANcited by 635
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  6. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  7. 2017Leveraging Demonstrations for Deep Reinforcement Learning on Robotics Problems with Sparse Rewardscited by 302
  8. 2016NIPS 2016 Tutorial: Generative Adversarial Networkscited by 284
  9. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  10. 2017Noisy Networks for Explorationcited by 262
  11. 2016Sample Efficient Actor-Critic with Experience Replaycited by 185
  12. 2016A Connection between Generative Adversarial Networks, Inverse Reinforcement Learning, and Energy-Based Modelscited by 167
  13. 2017Parameter Space Noise for Explorationcited by 158
  14. 2016Energy-based Generative Adversarial Networkcited by 119
  15. 2016Learning values across many orders of magnitudecited by 80
  16. 2016Q-Prop: Sample-Efficient Policy Gradient with An Off-Policy Criticcited by 77
  17. 2017Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Netscited by 64
  18. 2016Model-Free Imitation Learning with Policy Optimizationcited by 52
  19. 2016Connecting Generative Adversarial Networks and Actor-Critic Methodscited by 49
  20. 2017Many Paths to Equilibrium: GANs Do Not Need to Decrease a Divergence At Every Stepcited by 45
  21. 2017InfoGAIL: Interpretable Imitation Learning from Visual Demonstrationscited by 11
  22. 2017Burn-In Demonstrations for Multi-Modal Imitation Learningcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2019: 6; 2020: 4; 2021: 7; 2022: 3; 2023: 8; 2024: 5; 2025: 1; 2026: 2.

36 papers.

Sort:

By area:

  1. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 193
  2. 2019Goal-conditioned Imitation Learningcited by 159
  3. 2019Imitation Learning as f-Divergence Minimizationcited by 121
  4. 2019State Alignment-based Imitation Learningcited by 73
  5. 2020Strictly Batch Imitation Learning by Energy-based Distribution Matchingcited by 54
  6. 2019Task-Relevant Adversarial Imitation Learningcited by 45
  7. 2021Visual Adversarial Imitation Learning using Variational Modelscited by 42
  8. 2019Positive-Unlabeled Reward Learningcited by 36
  9. 2020Imitation Learning with Sinkhorn Distancescited by 30
  10. 2021Online Apprenticeship Learningcited by 28
  11. 2021Learning by Watchingcited by 22
  12. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learningcited by 15
24 more
  1. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  2. 2023A Survey of Demonstration Learningcited by 12
  3. 2024Efficient Imitation Learning with Conservative World Modelscited by 11
  4. 2023Adversarial Imitation Learning from Visual Observations using Latent Informationcited by 10
  5. 2021A Pragmatic Look at Deep Imitation Learningcited by 9
  6. 2024Visually Robust Adversarial Imitation Learning from Videos with Contrastive Learningcited by 9
  7. 2022Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulationcited by 7
  8. 2023Mimicking Better by Matching the Approximate Action Distributioncited by 5
  9. 2020Forward and inverse reinforcement learning sharing network weights and hyperparameterscited by 4
  10. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspectivecited by 4
  11. 2022Latent Policies for Adversarial Imitation Learningcited by 3
  12. 2023Deep Reinforcement Learning with Explicit Context Representationcited by 3
  13. 2023Explaining by Imitating: Understanding Decisions by Interpretable Policy Learningcited by 3
  14. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  15. 2024Towards Sample-Efficiency and Generalization of Transfer and Inverse Reinforcement Learning: A Comprehensive Literature Reviewcited by 3
  16. 2023Conditional Kernel Imitation Learning for Continuous State Environmentscited by 2
  17. 2021Sample Efficient Imitation Learning via Reward Function Trained in Advancecited by 1
  18. 2022Model-Based Imitation Learning Using Entropy Regularization of Model and Policycited by 1
  19. 2023Quantum Imitation Learningcited by 1
  20. 2026Learning from Demonstrations via Capability-Aware Goal Samplingcited by 1
  21. 2026Data-Efficient Hierarchical Goal-Conditioned Reinforcement Learning via Normalizing Flowscited by 1
  22. 2019Relational Mimic for Visual Adversarial Imitation Learningcited by 0
  23. 2024Inverse Delayed Reinforcement Learningcited by 0
  24. 2025Noise-Guided Transport for Imitation Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  2. 2024C-GAIL: Stabilizing Generative Adversarial Imitation Learning with Control Theory
  3. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
  4. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate
  5. 2021Provably Efficient Generative Adversarial Imitation Learning for Online and Offline Setting with Linear Function Approximation
  6. 2023Sample-efficient Adversarial Imitation Learning
  7. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  8. 2024Provably Efficient Off-Policy Adversarial Imitation Learning with Convergence Guarantees