mlsanityData to 14 Oct 2026

Auto-Encoding Adversarial Imitation Learning

arXiv 2206.11004 ↗First posted 22 Jun 2022cs.LGcited by 1

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Reinforcement learning (RL) provides a powerful framework for decision-making, but its application in practice often requires a carefully designed reward function. Adversarial Imitation Learning (AIL) sheds light on automatic policy acquisition without access to the reward signal from the environment. In this work, we propose Auto-Encoding Adversarial Imitation Learning (AEAIL), a robust and scalable AIL framework. To induce expert policies from demonstrations, AEAIL utilizes the reconstruction error of an auto-encoder as a reward signal, which provides more information for optimizing policies than the prior discriminator-based ones. Subsequently, we use the derived objective functions to train the auto-encoder and the agent policy. Experiments show that our AEAIL performs superior compared to state-of-the-art methods on both state and image based environments. More importantly, AEAIL shows much better robustness when the expert demonstrations are noisy.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 27 of its 57 references that are in the map. First posted on arXiv 22 Jun 2022.

Builds on

27 of 57 references.

Closest ancestors · 11

Same direction, newest first

  1. 2021IQ-Learn: Inverse soft-Q Learning for Imitation23 Jun
  2. 2021What Matters for Adversarial Imitation Learning?1 Jun
  3. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning2 Oct
  4. 2020When Will Generative Adversarial Imitation Learning Algorithms Attain Global Convergence24 Jun
  5. 2020Primal Wasserstein Imitation Learning8 Jun
  6. 2020State-only Imitation with Transition Dynamics Mismatch27 Feb
  7. 2019Imitation Learning via Off-Policy Distribution Matching10 Dec
  8. 2019A Divergence Minimization Perspective on Imitation Learning Methods6 Nov
  9. 2019Random Expert Distillation: Imitation Learning via Expert Policy Support Estimation16 May
  10. 2019Imitation Learning from Imperfect Demonstration27 Jan
  11. 2018Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning9 Sep

Other references · 16

  1. 2015Trust Region Policy Optimizationcited by 2,580
  2. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  3. 2016Generative Adversarial Imitation Learningcited by 1,597
  4. 2017Wasserstein GANcited by 639
  5. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 490
  6. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  7. 2019Extrapolating Beyond Suboptimal Demonstrations via Inverse Reinforcement Learning from Observationscited by 260
  8. 2018Distributed Distributional Deterministic Policy Gradientscited by 199
  9. 2015Learning Continuous Control Policies by Stochastic Value Gradientscited by 162
  10. 2017BEGAN: Boundary Equilibrium Generative Adversarial Networkscited by 121
  11. 2016Energy-based Generative Adversarial Networkcited by 119
  12. 2020Learning to Utilize Shaping Rewards: A New Approach of Reward Shapingcited by 114
4 more
  1. 2017Third-Person Imitation Learningcited by 103
  2. 2016Transfer from Simulation to Real World through Learning Deep Inverse Dynamics Modelcited by 54
  3. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  4. 2019Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidancecited by 25

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024LASIL: Learner-Aware Supervised Imitation Learning For Long-term Microscopic Traffic Simulationcited by 8

Similar, not cited

Close in wording, no citation either way

  1. 2025SD2AIL: Adversarial Imitation Learning from Synthetic Demonstrations via Diffusion Models
  2. 2026Adversarial Imitation Learning with General Function Approximation: Theoretical Analysis and Practical Algorithms
  3. 2020Off-Policy Adversarial Inverse Reinforcement Learning
  4. 2026Provably Efficient Policy-Reward Co-Pretraining for Adversarial Imitation Learning
  5. 2023Policy Contrastive Imitation Learning
  6. 2020ADAIL: Adaptive Adversarial Imitation Learning
  7. 2019Adversarial Imitation Learning from Incomplete Demonstrations
  8. 2024Diffusion-Reward Adversarial Imitation Learning