# Robust Imitation of Diverse Behaviors

arXiv: https://arxiv.org/abs/1707.02747 · HTML: https://mlsanity.com/paper/1707.02747/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jul 2017.

## Built on by (most cited first, 30 of 86)

- [RT-H: Action Hierarchies Using Language](https://mlsanity.com/paper/2403.01823/) (2024)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [OPAL: Offline Primitive Discovery for Accelerating Offline Reinforcement Learning](https://mlsanity.com/paper/2010.13611/) (2020)
- [Reinforcement and Imitation Learning for Diverse Visuomotor Skills](https://mlsanity.com/paper/1802.09564/) (2018)
- [SimPoE: Simulated Character Control for 3D Human Pose Estimation](https://mlsanity.com/paper/2104.00683/) (2021)
- [Adversarial Motion Priors Make Good Substitutes for Complex Reward Functions](https://mlsanity.com/paper/2203.15103/) (2022)
- [A Survey of Deep RL and IL for Autonomous Driving Policy Learning](https://mlsanity.com/paper/2101.01993/) (2021)
- [Imitation Learning: Progress, Taxonomies and Challenges](https://mlsanity.com/paper/2106.12177/) (2021)
- [Multi-Agent Imitation Learning for Driving Simulation](https://mlsanity.com/paper/1803.01044/) (2018)
- [Learning Agile Skills via Adversarial Imitation of Rough Partial Demonstrations](https://mlsanity.com/paper/2206.11693/) (2022)
- [Imitate and Repurpose: Learning Reusable Robot Movement Skills From Human and Animal Behaviors](https://mlsanity.com/paper/2203.17138/) (2022)
- [Robust Imitation Learning against Variations in Environment Dynamics](https://mlsanity.com/paper/2206.09314/) (2022)
- [Learning from Demonstration in the Wild](https://mlsanity.com/paper/1811.03516/) (2018)
- [MoCapAct: A Multi-Task Dataset for Simulated Humanoid Control](https://mlsanity.com/paper/2208.07363/) (2022)
- [Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems](https://mlsanity.com/paper/2102.07659/) (2021)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)
- [Multi-task Maximum Entropy Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.08882/) (2018)
- [Discovering Diverse Solutions in Deep Reinforcement Learning by Maximizing State-Action-Based Mutual Information](https://mlsanity.com/paper/2103.07084/) (2021)
- [AMP: Adversarial Motion Priors for Stylized Physics-Based Character Control](https://mlsanity.com/paper/2104.02180/) (2021)
- [Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration](https://mlsanity.com/paper/2108.06038/) (2021)
- [Collect & Infer -- a fresh look at data-efficient Reinforcement Learning](https://mlsanity.com/paper/2108.10273/) (2021)
- [A Survey of Imitation Learning Methods, Environments and Metrics](https://mlsanity.com/paper/2404.19456/) (2024)
- [Lipschitzness Is All You Need To Tame Off-policy Generative Adversarial Imitation Learning](https://mlsanity.com/paper/2006.16785/) (2020)
- [Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations](https://mlsanity.com/paper/2203.02107/) (2022)
- [A Survey on Imitation Learning for Contact-Rich Tasks in Robotics](https://mlsanity.com/paper/2506.13498/) (2025)
- [Imitation by Predicting Observations](https://mlsanity.com/paper/2107.03851/) (2021)
- [VILD: Variational Imitation Learning with Diverse-quality Demonstrations](https://mlsanity.com/paper/1909.06769/) (2019)
- [Inverse Contextual Bandits: Learning How Behavior Evolves over Time](https://mlsanity.com/paper/2107.06317/) (2021)
- [Triple-GAIL: A Multi-Modal Imitation Learning Framework with Generative Adversarial Nets](https://mlsanity.com/paper/2005.10622/) (2020)
- [Adversarial Imitation Learning with Trajectorial Augmentation and Correction](https://mlsanity.com/paper/2103.13887/) (2021)

## Abstract (arXiv)

Deep generative models have recently shown great promise in imitation learning for motor control. Given enough data, even supervised approaches can do one-shot imitation learning; however, they are vulnerable to cascading failures when the agent trajectory diverges from the demonstrations. Compared to purely supervised methods, Generative Adversarial Imitation Learning (GAIL) can learn more robust controllers from fewer demonstrations, but is inherently mode-seeking and more difficult to train. In this paper, we show how to combine the favourable aspects of these two approaches. The base of our model is a new type of variational autoencoder on demonstration trajectories that learns semantic policy embeddings. We show that these embeddings can be learned on a 9 DoF Jaco robot arm in reaching tasks, and then smoothly interpolated with a resulting smooth interpolation of reaching behavior. Leveraging these policy representations, we develop a new version of GAIL that (1) is much more robust than the purely-supervised controller, especially with few demonstrations, and (2) avoids mode collapse, capturing many diverse behaviors when GAIL on its own does not. We demonstrate our approach on learning diverse gaits from demonstration on a 2D biped and a 62 DoF 3D humanoid in the MuJoCo physics environment.