mlsanityData to 7 Oct 2026

Goal-Aware Generative Adversarial Imitation Learning from Imperfect Demonstration for Robotic Cloth Manipulation

arXiv 2209.10149 ↗First posted 21 Sep 2022cs.ROcited by 7

On the map: Offline RL and imitation learning from data → Imitation learning methods

Abstract (arXiv)

Generative Adversarial Imitation Learning (GAIL) can learn policies without explicitly defining the reward function from demonstrations. GAIL has the potential to learn policies with high-dimensional observations as input, e.g., images. By applying GAIL to a real robot, perhaps robot policies can be obtained for daily activities like washing, folding clothes, cooking, and cleaning. However, human demonstration data are often imperfect due to mistakes, which degrade the performance of the resulting policies. We address this issue by focusing on the following features: 1) many robotic tasks are goal-reaching tasks, and 2) labeling such goal states in demonstration data is relatively easy. With these in mind, this paper proposes Goal-Aware Generative Adversarial Imitation Learning (GA-GAIL), which trains a policy by introducing a second discriminator to distinguish the goal state in parallel with the first discriminator that indicates the demonstration data. This extends a standard GAIL framework to more robustly learn desirable policies even from imperfect demonstrations through a goal-state discriminator that promotes achieving the goal state. Furthermore, GA-GAIL employs the Entropy-maximizing Deep P-Network (EDPN) as a generator, which considers both the smoothness and causal entropy in the policy update, to achieve stable policy learning from two discriminators. Our proposed method was successfully applied to two real-robotic cloth-manipulation tasks: turning a handkerchief over and folding clothes. We confirmed that it learns cloth-manipulation policies without task-specific reward function design. Video of the real experiments are available at https://youtu.be/h_nII2ooUrE.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Imitation learning methods, one of 13 directions in Offline RL and imitation learning from data. Builds on 31 of its 57 references that are in the map. First posted on arXiv 21 Sep 2022.

Builds on

31 of 57 references.

Closest ancestors · 10

Same direction, newest first

  1. 2020Robust Imitation Learning from Noisy Demonstrations20 Oct
  2. 2020Generative Adversarial Imitation Learning with Neural Networks: Global Optimality and Convergence Rate8 Mar
  3. 2019Integration of Imitation Learning using GAIL and Reinforcement Learning using Task-achievement Rewards via Probabilistic Graphical Model3 Jul
  4. 2019Goal-conditioned Imitation Learning13 Jun
  5. 2019Adversarial Imitation Learning from Incomplete Demonstrations29 May
  6. 2019Generative predecessor models for sample-efficient imitation learning1 Apr
  7. 2019Imitation Learning from Imperfect Demonstration27 Jan
  8. 2018Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow1 Oct
  9. 2018Sample-Efficient Imitation Learning via Generative Adversarial Nets6 Sep
  10. 2018Reinforcement Learning from Imperfect Demonstrations14 Feb

Other references · 21

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  3. 2015Trust Region Policy Optimizationcited by 2,503
  4. 2016Generative Adversarial Imitation Learningcited by 1,576
  5. 2017Hindsight Experience Replaycited by 1,096
  6. 2017Reinforcement Learning with Deep Energy-Based Policiescited by 659
  7. 2017Learning Robust Rewards with Adversarial Inverse Reinforcement Learningcited by 485
  8. 2018Data-Efficient Hierarchical Reinforcement Learningcited by 436
  9. 2017FeUdal Networks for Hierarchical Reinforcement Learningcited by 328
  10. 2016Hierarchical Deep Reinforcement Learning: Integrating Temporal Abstraction and Intrinsic Motivationcited by 282
  11. 2018Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Researchcited by 256
  12. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
9 more
  1. 2018Sim-to-Real Reinforcement Learning for Deformable Object Manipulationcited by 124
  2. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  3. 2018Hierarchical Imitation and Reinforcement Learningcited by 88
  4. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  5. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  6. 2017Learning human behaviors from motion capture by adversarial imitationcited by 75
  7. 2018Gotta Learn Fast: A New Benchmark for Generalization in RLcited by 59
  8. 2018Adversarial Imitation via Variational Inverse Reinforcement Learningcited by 36
  9. 2019Dynamic Cloth Manipulation with Deep Reinforcement Learningcited by 26

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 3.

7 papers.

  1. 20243D Diffuser Actor: Policy Diffusion with 3D Scene Representationscited by 278
  2. 2025Dexterous Manipulation through Imitation Learning: A Surveycited by 18
  3. 2024Manipulation Facing Threats: Evaluating Physical Vulnerabilities in End-to-End Vision Language Action Modelscited by 16
  4. 2023Graph learning in robotics: a surveycited by 5
  5. 2025Time-Unified Diffusion Policy with Action Discrimination for Robotic Manipulationcited by 5
  6. 2023Leveraging Optimal Transport for Enhanced Offline Reinforcement Learning in Surgical Robotic Environmentscited by 3
  7. 2025Model Predictive Adversarial Imitation Learning for Planning from Observationcited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2025Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL) for Multi-goal Robotic Manipulation Tasks
  2. 2020On Computation and Generalization of Generative Adversarial Imitation Learning
  3. 2022Latent Policies for Adversarial Imitation Learning
  4. 2020f-GAIL: Learning f-Divergence for Generative Adversarial Imitation Learning
  5. 2023Exploring Gradient Explosion in Generative Adversarial Imitation Learning: A Probabilistic Perspective
  6. 2019Situated GAIL: Multitask imitation using task-conditioned adversarial inverse reinforcement learning
  7. 2022Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations
  8. 2019Task-Relevant Adversarial Imitation Learning