mlsanityData to 7 Oct 2026

What Matters for Adversarial Imitation Learning?

arXiv 2106.00672 ↗First posted 1 Jun 2021cs.LG cs.AI cs.NEcited by 70

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Adversarial imitation learning has become a popular framework for imitation in continuous control. Over the years, several variations of its components were proposed to enhance the performance of the learned policies as well as the sample complexity of the algorithm. In practice, these choices are rarely tested all together in rigorous empirical studies. It is therefore difficult to discuss and understand what choices, among the high-level algorithmic options as well as low-level implementation details, matter. To tackle this issue, we implement more than 50 of these choices in a generic adversarial imitation learning framework and investigate their impacts in a large-scale study (>500k trained agents) with both synthetic and human-generated demonstrations. While many of our findings confirm common practices, some of them are surprising or even contradict prior work. In particular, our results suggest that artificial demonstrations are not a good proxy for human data and that the very common practice of evaluating imitation algorithms only with synthetic demonstrations may lead to algorithms which perform poorly in the more realistic scenarios with human demonstrations.

Key facts. Cited by 70 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 37 of its 54 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

37 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2020Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning28 Jun
  2. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  3. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 34

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2020Scaling Laws for Neural Language Modelscited by 6,411
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  6. 2015Trust Region Policy Optimizationcited by 2,503
  7. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  8. 2017Improved Training of Wasserstein GANscited by 2,223
  9. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 1,916
  10. 2017Decoupled Weight Decay Regularizationcited by 1,722
  11. 2016OpenAI Gymcited by 1,684
  12. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
22 more
  1. 2016Generative Adversarial Imitation Learningcited by 1,576
  2. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  3. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,179
  4. 2019Dota 2 with Large Scale Deep Reinforcement Learningcited by 1,084
  5. 2020Designing Network Design Spacescited by 954
  6. 2018Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representationscited by 929
  7. 2017A Distributional Perspective on Reinforcement Learningcited by 743
  8. 2017Deep Reinforcement Learning that Matterscited by 727
  9. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  10. 2018Learning Dexterous In-Hand Manipulationcited by 643
  11. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  12. 2017Are GANs Created Equal? A Large-Scale Studycited by 267
  13. 2018Distributed Distributional Deterministic Policy Gradientscited by 195
  14. 2019Benchmarking Model-Based Reinforcement Learningcited by 158
  15. 2020What Matters In On-Policy Reinforcement Learning? A Large-Scale Empirical Studycited by 157
  16. 2020Acme: A Research Framework for Distributed Reinforcement Learningcited by 146
  17. 2017Reproducibility of Benchmarked Deep Reinforcement Learning Tasks for Continuous Controlcited by 92
  18. 2017Data-efficient Deep Reinforcement Learning for Dexterous Manipulationcited by 61
  19. 2018The Mirage of Action-Dependent Baselines in Reinforcement Learningcited by 46
  20. 2019Making Efficient Use of Demonstrations to Solve Hard Exploration Problemscited by 45
  21. 2019Positive-Unlabeled Reward Learningcited by 36
  22. 2020Batch Exploration with Examples for Scalable Robotic Reinforcement Learningcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 16; 2023: 15; 2024: 15; 2025: 5; 2026: 13.

70 papers.

Sort:

By area:

  1. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  2. 2023A Survey of Imitation Learning: Algorithms, Recent Developments, and Challengescited by 145
  3. 2024Equivariant Diffusion Policycited by 61
  4. 2022Invariance in Policy Optimisation and Partial Identifiability in Reward Learningcited by 46
  5. 2023Optimal Transport for Offline Imitation Learningcited by 42
  6. 2023AlignDiff: Aligning Diverse Human Preferences via Behavior-Customisable Diffusion Modelcited by 37
  7. 2022imitation: Clean Imitation Learning Implementationscited by 34
  8. 2022Misspecification in Inverse Reinforcement Learningcited by 27
  9. 2024Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learningcited by 27
  10. 2021Continuous Control with Action Quantization from Demonstrationscited by 21
  11. 2023Coherent Soft Imitation Learningcited by 21
  12. 2022Planning for Sample Efficient Imitation Learningcited by 20
38 more
  1. 2022A Ranking Game for Imitation Learningcited by 18
  2. 2024Diffusion-Reward Adversarial Imitation Learningcited by 18
  3. 2021OPIRL: Sample Efficient Off-Policy Inverse Reinforcement Learning via Distribution Matchingcited by 15
  4. 2022Rethinking ValueDice: Does It Really Improve Performance?cited by 15
  5. 2022Learn what matters: cross-domain imitation learning with task-relevant embeddingscited by 14
  6. 2022Understanding Adversarial Imitation Learning in Small Sample Regime: A Stage-coupled Analysiscited by 13
  7. 2024Supervised Fine-Tuning as Inverse Reinforcement Learningcited by 12
  8. 2021Braxlines: Fast and Interactive Toolkit for RL-driven Behavior Engineering beyond Reward Maximizationcited by 11
  9. 2022Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Taskscited by 10
  10. 2021A Pragmatic Look at Deep Imitation Learningcited by 9
  11. 2023Multimodal and Force-Matched Imitation Learning with a See-Through Visuotactile Sensorcited by 9
  12. 2024DrS: Learning Reusable Dense Rewards for Multi-Stage Taskscited by 9
  13. 2022Learning Energy Networks with Generalized Fenchel-Young Lossescited by 8
  14. 2022An Evaluation Study of Intrinsic Motivation Techniques applied to Reinforcement Learning over Hard Exploration Environmentscited by 8
  15. 2024A Comparison of Imitation Learning Algorithms for Bimanual Manipulationcited by 8
  16. 2023BC-IRL: Learning Generalizable Reward Functions from Demonstrationscited by 7
  17. 2025Inverse Reinforcement Learning Meets Large Language Model Post-Training: Basics, Advances, and Opportunitiescited by 7
  18. 2022Inferring Versatile Behavior from Demonstrations by Matching Geometric Descriptorscited by 6
  19. 2024OLLIE: Imitation Learning from Offline Pretraining to Online Finetuningcited by 6
  20. 2023Curricular Subgoals for Inverse Reinforcement Learningcited by 5
  21. 2024Adversarial Imitation Learning via Boostingcited by 5
  22. 2025IL-SOAR : Imitation Learning with Soft Optimistic Actor cRiticcited by 5
  23. 2023Get Back Here: Robust Imitation by Return-to-Distribution Planningcited by 4
  24. 2024Quantifying the Sensitivity of Inverse Reinforcement Learning to Misspecificationcited by 4
  25. 2024Partial Identifiability and Misspecification in Inverse Reinforcement Learningcited by 4
  26. 2021Learning from Guided Play: A Scheduled Hierarchical Approach for Improving Exploration in Adversarial Imitation Learningcited by 3
  27. 2022Latent Policies for Adversarial Imitation Learningcited by 3
  28. 2022Co-Imitation: Learning Design and Behaviour by Imitationcited by 3
  29. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learningcited by 3
  30. 2023Curriculum-Based Imitation of Versatile Skillscited by 3
  31. 2023Policy Contrastive Imitation Learningcited by 3
  32. 2024BeTAIL: Behavior Transformer Adversarial Imitation Learning from Human Racing Gameplaycited by 3
  33. 2024Enhancing Inverse Reinforcement Learning through Encoding Dynamic Information in Reward Shapingcited by 3
  34. 2021On the Benefits of Inducing Local Lipschitzness for Robust Generative Adversarial Imitation Learningcited by 2
  35. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2
  36. 2022Auto-Encoding Adversarial Imitation Learningcited by 1
  37. 2022Ask-AC: An Initiative Advisor-in-the-Loop Actor-Critic Frameworkcited by 1
  38. 2023Using Offline Data to Speed Up Reinforcement Learning in Procedurally Generated Environmentscited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances
  2. 2021Imitation Learning: Progress, Taxonomies and Challenges
  3. 2023Sample-efficient Adversarial Imitation Learning
  4. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
  5. 2025Learning What to Do and What Not To Do: Offline Imitation from Expert and Undesirable Demonstrations
  6. 2022Robust Learning from Observation with Model Misspecification
  7. 2023Adversarial Imitation Learning from Visual Observations using Latent Information
  8. 2018Generative Adversarial Imitation from Observation