# Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulation

arXiv: https://arxiv.org/abs/2209.10149 · HTML: https://mlsanity.com/paper/2209.10149/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 31 of its 57 references that are in the map.
- First posted on arXiv 21 Sep 2022.

## Placed in

- Offline RL and imitation learning from data / [Imitation learning methods](https://mlsanity.com/offlinerl/imitation-learning-methods/)

## Builds on (most cited first, 30 of 31)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [Hindsight Experience Replay](https://mlsanity.com/paper/1707.01495/) (2017)
- [Reinforcement Learning with Deep Energy-Based Policies](https://mlsanity.com/paper/1702.08165/) (2017)
- [Learning Robust Rewards with Adversarial Inverse Reinforcement Learning](https://mlsanity.com/paper/1710.11248/) (2017)
- [Data-Efficient Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1805.08296/) (2018)
- [FeUdal Networks for Hierarchical Reinforcement Learning](https://mlsanity.com/paper/1703.01161/) (2017)
- [Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivation](https://mlsanity.com/paper/1604.06057/) (2016)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [Deep Dynamics Models for Learning Dexterous Manipulation](https://mlsanity.com/paper/1909.11652/) (2019)
- [Goal-conditioned Imitation Learning](https://mlsanity.com/paper/1906.05838/) (2019)
- [Sim-to-Real Reinforcement Learning for Deformable Object Manipulation](https://mlsanity.com/paper/1806.07851/) (2018)
- [The Ingredients of Real-World Robotic Reinforcement Learning](https://mlsanity.com/paper/2004.12570/) (2020)
- [Imitation Learning from Imperfect Demonstration](https://mlsanity.com/paper/1901.09387/) (2019)
- [Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow](https://mlsanity.com/paper/1810.00821/) (2018)
- [Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learning](https://mlsanity.com/paper/1711.06782/) (2017)
- [Hierarchical Imitation and Reinforcement Learning](https://mlsanity.com/paper/1803.00590/) (2018)
- [Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Cost](https://mlsanity.com/paper/1810.06045/) (2018)
- [Learning human behaviors from motion capture by adversarial imitation](https://mlsanity.com/paper/1707.02201/) (2017)
- [Reinforcement Learning from Imperfect Demonstrations](https://mlsanity.com/paper/1802.05313/) (2018)
- [Gotta Learn Fast: A New Benchmark for Generalization in RL](https://mlsanity.com/paper/1804.03720/) (2018)
- [Sample-Efficient Imitation Learning via Generative Adversarial Nets](https://mlsanity.com/paper/1809.02064/) (2018)
- [Adversarial Imitation via Variational Inverse Reinforcement Learning](https://mlsanity.com/paper/1809.06404/) (2018)
- [Dynamic Cloth Manipulation with Deep Reinforcement Learning](https://mlsanity.com/paper/1910.14475/) (2019)
- [Robust Imitation Learning from Noisy Demonstrations](https://mlsanity.com/paper/2010.10181/) (2020)
- [Adversarial Imitation Learning from Incomplete Demonstrations](https://mlsanity.com/paper/1905.12310/) (2019)
- [Generative predecessor models for sample-efficient imitation learning](https://mlsanity.com/paper/1904.01139/) (2019)
- [Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Model](https://mlsanity.com/paper/1907.02140/) (2019)

## Built on by (most cited first, 7 of 7)

- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [Dexterous Manipulation through Imitation Learning: A Survey](https://mlsanity.com/paper/2504.03515/) (2025)
- [Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Models](https://mlsanity.com/paper/2409.13174/) (2024)
- [Graph learning in robotics: a survey](https://mlsanity.com/paper/2310.04294/) (2023)
- [Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulation](https://mlsanity.com/paper/2506.09422/) (2025)
- [Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environments](https://mlsanity.com/paper/2310.08841/) (2023)
- [Model Predictive Adversarial Imitation Learning for Planning from Observation](https://mlsanity.com/paper/2507.21533/) (2025)

## Abstract (arXiv)

Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: 1) many robotic tasks are goal-reaching tasks, and 2) labeling such goal states in demonstration data is relatively easy. With these in mind, this paper proposes Goal-Aware Generative Adversarial Imitation Learning (GA-GAIL), which trains a policy by introducing a second discriminator to distinguish the goal state in parallel with the first discriminator that indicates the demonstration data. This extends a standard GAIL framework to more robustly learn desirable policies even from imperfect demonstrations through a goal-state discriminator that promotes achieving the goal state. Furthermore, GA-GAIL employs the Entropy-maximizing Deep P-Network (EDPN) as a generator, which considers both the smoothness and causal entropy in the policy update, to achieve stable policy learning from two discriminators. Our proposed method was successfully applied to two real-robotic cloth-manipulation tasks: turning a handkerchief over and folding clothes. We confirmed that it learns cloth-manipulation policies without task-specific reward function design. Video of the real experiments are available at https://youtu.be/h_nII2ooUrE.