# Multi-Modal Imitation Learning from Unstructured Demonstrations using Generative Adversarial Nets

arXiv: https://arxiv.org/abs/1705.10479 · HTML: https://mlsanity.com/paper/1705.10479/

## Key facts

- Cited by 64 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2017.

## Built on by (most cited first, 30 of 64)

- [Behavior Transformers: Cloning k modes with one stone](https://mlsanity.com/paper/2206.11251/) (2022)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scale](https://mlsanity.com/paper/2104.08212/) (2021)
- [Language Conditioned Imitation Learning over Unstructured Data](https://mlsanity.com/paper/2005.07648/) (2020)
- [Imitation Learning via Off-Policy Distribution Matching](https://mlsanity.com/paper/1912.05032/) (2019)
- [Imitation Learning as f-Divergence Minimization](https://mlsanity.com/paper/1905.12888/) (2019)
- [Meta-Inverse Reinforcement Learning with Probabilistic Context Variables](https://mlsanity.com/paper/1909.09314/) (2019)
- [Imitation Learning by Estimating Expertise of Demonstrators](https://mlsanity.com/paper/2202.01288/) (2022)
- [Visual Adversarial Imitation Learning using Variational Models](https://mlsanity.com/paper/2107.08829/) (2021)
- [Learning Multimodal Rewards from Rankings](https://mlsanity.com/paper/2109.12750/) (2021)
- [Learning a Prior over Intent via Meta-Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.12573/) (2018)
- [Sample-Efficient Imitation Learning via Generative Adversarial Nets](https://mlsanity.com/paper/1809.02064/) (2018)
- [One-Shot Hierarchical Imitation Learning of Compound Visuomotor Tasks](https://mlsanity.com/paper/1810.11043/) (2018)
- [Generative Adversarial Self-Imitation Learning](https://mlsanity.com/paper/1812.00950/) (2018)
- [MORAL: Aligning AI with Human Norms through Multi-Objective Reinforced Active Learning](https://mlsanity.com/paper/2201.00012/) (2021)
- [Eliciting Compatible Demonstrations for Multi-Human Imitation Learning](https://mlsanity.com/paper/2210.08073/) (2022)
- [Multi-task Maximum Entropy Inverse Reinforcement Learning](https://mlsanity.com/paper/1805.08882/) (2018)
- [An Independently Learnable Hierarchical Model for Bilateral Control-Based Imitation Learning Applications](https://mlsanity.com/paper/2203.08619/) (2022)
- [STEER: Flexible Robotic Manipulation via Dense Language Grounding](https://mlsanity.com/paper/2411.03409/) (2024)
- [Recent Advances in Leveraging Human Guidance for Sequential Decision-Making Tasks](https://mlsanity.com/paper/2107.05825/) (2021)
- [VILD: Variational Imitation Learning with Diverse-quality Demonstrations](https://mlsanity.com/paper/1909.06769/) (2019)
- [Maximum Causal Tsallis Entropy Imitation Learning](https://mlsanity.com/paper/1805.08336/) (2018)
- [Learning from Guided Play: Improving Exploration for Adversarial Imitation Learning with Simple Auxiliary Tasks](https://mlsanity.com/paper/2301.00051/) (2022)
- [Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Models](https://mlsanity.com/paper/2503.23271/) (2025)
- [Intention-Conditioned Flow Occupancy Models](https://mlsanity.com/paper/2506.08902/) (2025)
- [VFP: Variational Flow-Matching Policy for Multi-Modal Robot Manipulation](https://mlsanity.com/paper/2508.01622/) (2025)
- [CompILE: Compositional Imitation Learning and Execution](https://mlsanity.com/paper/1812.01483/) (2018)
- [Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Model](https://mlsanity.com/paper/1907.02140/) (2019)
- [Crowd-PrefRL: Preference-Based Reward Learning from Crowds](https://mlsanity.com/paper/2401.10941/) (2024)
- [Deep Adaptive Multi-Intention Inverse Reinforcement Learning](https://mlsanity.com/paper/2107.06692/) (2021)

## Abstract (arXiv)

Imitation learning has traditionally been applied to learn a single task from demonstrations thereof. The requirement of structured and isolated demonstrations limits the scalability of imitation learning approaches as they are difficult to apply to real-world scenarios, where robots have to be able to execute a multitude of tasks. In this paper, we propose a multi-modal imitation learning framework that is able to segment and imitate skills from unlabelled and unstructured demonstrations by learning skill segmentation and imitation learning jointly. The extensive simulation results indicate that our method can efficiently separate the demonstrations into individual skills and learn to imitate them using a single multi-modal policy. The video of our experiments is available at http://sites.google.com/view/nips17intentiongan