# Adversarial Imitation Learning from Incomplete Demonstrations

arXiv: https://arxiv.org/abs/1905.12310 · HTML: https://mlsanity.com/paper/1905.12310/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 13 of its 40 references that are in the map.
- First posted on arXiv 29 May 2019.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 13 of 13)

- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Diversity is All You Need: Learning Skills without a Reward Function](https://mlsanity.com/paper/1802.06070/) (2018)
- [Behavioral Cloning from Observation](https://mlsanity.com/paper/1805.01954/) (2018)
- [Guided Cost Learning: Deep Inverse Optimal Control via Policy Optimization](https://mlsanity.com/paper/1603.00448/) (2016)
- [Third-Person Imitation Learning](https://mlsanity.com/paper/1703.01703/) (2017)
- [Learning human behaviors from motion capture by adversarial imitation](https://mlsanity.com/paper/1707.02201/) (2017)
- [Reinforcement Learning from Imperfect Demonstrations](https://mlsanity.com/paper/1802.05313/) (2018)
- [On the Complexity of Solving Markov Decision Problems](https://mlsanity.com/paper/1302.4971/) (2013)
- [Learning Transferable Policies for Monocular Reactive MAV Control](https://mlsanity.com/paper/1608.00627/) (2016)

## Built on by (most cited first, 16 of 16)

- [Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing](https://mlsanity.com/paper/2103.04909/) (2021)
- [You Only Live Once: Single-Life Reinforcement Learning](https://mlsanity.com/paper/2210.08863/) (2022)
- [Causal Navigation by Continuous-time Neural Networks](https://mlsanity.com/paper/2106.08314/) (2021)
- [Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulation](https://mlsanity.com/paper/2209.10149/) (2022)
- [In-Trajectory Inverse Reinforcement Learning: Learn Incrementally Before An Ongoing Trajectory Terminates](https://mlsanity.com/paper/2410.15612/) (2024)
- [Forward and inverse reinforcement learning sharing network weights and hyperparameters](https://mlsanity.com/paper/2008.07284/) (2020)
- [On the Guaranteed Almost Equivalence between Imitation Learning from Observation and Demonstration](https://mlsanity.com/paper/2010.08353/) (2020)
- [Are All Vision Models Created Equal? A Study of the Open-Loop to Closed-Loop Causality Gap](https://mlsanity.com/paper/2210.04303/) (2022)
- [Semi-Supervised Imitation Learning of Team Policies from Suboptimal Demonstrations](https://mlsanity.com/paper/2205.02959/) (2022)
- [Robust Offline Reinforcement Learning with Gradient Penalty and Constraint Relaxation](https://mlsanity.com/paper/2210.10469/) (2022)
- [SENSOR: Imitate Third-Person Expert's Behaviors via Active Sensoring](https://mlsanity.com/paper/2404.03386/) (2024)
- [PAIL: Performance based Adversarial Imitation Learning Engine for Carbon Neutral Optimization](https://mlsanity.com/paper/2407.08910/) (2024)
- [Mixture of Autoencoder Experts Guidance using Unlabeled and Incomplete Data for Exploration in Reinforcement Learning](https://mlsanity.com/paper/2507.15287/) (2025)
- [Investigating the Effects of Robot Engagement Communication on Learning from Demonstration](https://mlsanity.com/paper/2005.01020/) (2020)
- [Learning Time-Invariant Reward Functions through Model-Based Inverse Reinforcement Learning](https://mlsanity.com/paper/2107.03186/) (2021)
- [Divide and Repair: Using Options to Improve Performance of Imitation Learning Against Adversarial Demonstrations](https://mlsanity.com/paper/2306.04581/) (2023)

## Abstract (arXiv)

Imitation learning targets deriving a mapping from states to actions, a.k.a. policy, from expert demonstrations. Existing methods for imitation learning typically require any actions in the demonstrations to be fully available, which is hard to ensure in real applications. Though algorithms for learning with unobservable actions have been proposed, they focus solely on state information and overlook the fact that the action sequence could still be partially available and provide useful information for policy deriving. In this paper, we propose a novel algorithm called Action-Guided Adversarial Imitation Learning (AGAIL) that learns a policy from demonstrations with incomplete action sequences, i.e., incomplete demonstrations. The core idea of AGAIL is to separate demonstrations into state and action trajectories, and train a policy with state trajectories while using actions as auxiliary information to guide the training whenever applicable. Built upon the Generative Adversarial Imitation Learning, AGAIL has three components: a generator, a discriminator, and a guide. The generator learns a policy with rewards provided by the discriminator, which tries to distinguish state distributions between demonstrations and samples generated by the policy. The guide provides additional rewards to the generator when demonstrated actions for specific states are available. We compare AGAIL to other methods on benchmark tasks and show that AGAIL consistently delivers comparable performance to the state-of-the-art methods even when the action sequence in demonstrations is only partially available.