mlsanityData to 14 Oct 2026

Sample-efficient Adversarial Imitation Learning

arXiv 2303.07846 ↗First posted 14 Mar 2023cs.LG cs.AIcited by 0

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Imitation learning, in which learning is performed by demonstration, has been studied and advanced for sequential decision-making tasks in which a reward function is not predefined. However, imitation learning methods still require numerous expert demonstration samples to successfully imitate an expert's behavior. To improve sample efficiency, we utilize self-supervised representation learning, which can generate vast training signals from the given data. In this study, we propose a self-supervised representation-based adversarial imitation learning method to learn state and action representations that are robust to diverse distortions and temporally predictive, on non-image control tasks. In particular, in comparison with existing self-supervised learning methods for tabular data, we propose a different corruption method for state and action representations that is robust to diverse distortions. We theoretically and empirically observe that making an informative feature manifold with less sample complexity significantly improves the performance of imitation learning. The proposed method shows a 39% relative improvement over existing adversarial imitation learning methods on MuJoCo in a setting limited to 100 expert state-action pairs. Moreover, we conduct comprehensive ablations and additional experiments using demonstrations with varying optimality to provide insights into a range of factors.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 58 of its 80 references that are in the map. First posted on arXiv 14 Mar 2023.

Builds on

58 of 80 references.

Closest ancestors · 10

Same direction, newest first

  1. 2022Latent Policies for Adversarial Imitation Learning22 Jun
  2. 2021Confidence-Aware Imitation Learning from Demonstrations with Varying Optimality27 Oct
  3. 2020Robust Imitation Learning from Noisy Demonstrations20 Oct
  4. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning2 Oct
  5. 2020Adversarial Soft Advantage Fitting: Imitation Learning without Policy Optimization23 Jun
  6. 2020Safe Imitation Learning via Fast Bayesian Reward Inference from Preferences21 Feb
  7. 2020On Computation and Generalization of Generative Adversarial Imitation Learning9 Jan
  8. 2019Sample-efficient Adversarial Imitation Learning from Observation18 Jun
  9. 2019Imitation Learning from Imperfect Demonstration27 Jan
  10. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct

Other references · 48

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2017Proximal Policy Optimization Algorithmscited by 15,215
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,857
  11. 2015Trust Region Policy Optimizationcited by 2,580
  12. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,838
36 more
  1. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  2. 2016Generative Adversarial Imitation Learningcited by 1,597
  3. 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
  4. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  5. 2017Curiosity-driven Exploration by Self-supervised Predictioncited by 1,194
  6. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,057
  7. 2019AugMix: A Simple Data Processing Method to Improve Robustness and Uncertaintycited by 901
  8. 2020What Makes for Good Views for Contrastive Learning?cited by 846
  9. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
  12. 2019On Variational Bounds of Mutual Informationcited by 559
  13. 2020Reinforcement Learning with Augmented Datacited by 468
  14. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 378
  15. 2020Data-Efficient Reinforcement Learning with Self-Predictive Representationscited by 318
  16. 2018A Survey of Inverse Reinforcement Learning: Challenges, Methods and Progresscited by 298
  17. 2019Stochastic Latent Actor-Critic: Deep Reinforcement Learning with a Latent Variable Modelcited by 231
  18. 2020MixText: Linguistically-Informed Interpolation of Hidden Space for Semi-Supervised Text Classificationcited by 187
  19. 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 156
  20. 2015Variational Information Maximisation for Intrinsically Motivated Reinforcement Learningcited by 144
  21. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 136
  22. 2021Co-Mixup: Saliency Guided Joint Mixup with Supermodular Diversitycited by 128
  23. 2019Augmenting Data with Mixup for Sentence Classification: An Empirical Studycited by 125
  24. 2020Self-Supervised Policy Adaptation during Deploymentcited by 123
  25. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 122
  26. 2018A Kernel Theory of Modern Data Augmentationcited by 106
  27. 2021Stabilizing Deep Q-Learning with ConvNets and Vision Transformers under Data Augmentationcited by 105
  28. 2020Improving Generalization in Reinforcement Learning with Mixture Regularizationcited by 81
  29. 2020Deep Reinforcement and InfoMax Learningcited by 79
  30. 2020SuperMix: Supervising the Mixing Data Augmentationcited by 58
  31. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  32. 2019Data Augmentation Revisited: Rethinking the Distribution Gap between Clean and Augmented Datacited by 33
  33. 2022Does Self-supervised Learning Really Improve Reinforcement Learning from Pixels?cited by 24
  34. 2021Self-supervised Reinforcement Learning with Independently Controllable Subgoalscited by 16
  35. 2023SIRL: Similarity-based Implicit Representation Learningcited by 14
  36. 2018Inference Aided Reinforcement Learning for Incentive Mechanism Design in Crowdsourcingcited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Unlabeled Imperfect Demonstrations in Adversarial Imitation Learning
  2. 2021Deterministic and Discriminative Imitation (D2-Imitation): Revisiting Adversarial Imitation for Sample Efficiency
  3. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets
  4. 2018Injective State-Image Mapping facilitates Visual Adversarial Imitation Learning
  5. 2026Latent Wasserstein Adversarial Imitation Learning
  6. 2023Self-Supervised Adversarial Imitation Learning
  7. 2022Adversarial Imitation Learning from Video using a State Observer
  8. 2020State-only Imitation with Transition Dynamics Mismatch