# Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrations

arXiv: https://arxiv.org/abs/2203.02107 · HTML: https://mlsanity.com/paper/2203.02107/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 24 of its 33 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 24 of 24)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Generative Adversarial Imitation Learning](https://mlsanity.com/paper/1606.03476/) (2016)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [Transporter Networks: Rearranging the Visual World for Robotic Manipulation](https://mlsanity.com/paper/2010.14406/) (2020)
- [Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics](https://mlsanity.com/paper/1703.09312/) (2017)
- [Discriminator-Actor-Critic: Addressing Sample Inefficiency and Reward Bias in Adversarial Imitation Learning](https://mlsanity.com/paper/1809.02925/) (2018)
- [ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrations](https://mlsanity.com/paper/2107.14483/) (2021)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Self-Imitation Learning](https://mlsanity.com/paper/1806.05635/) (2018)
- [Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes](https://mlsanity.com/paper/1903.03911/) (2019)
- [A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representation](https://mlsanity.com/paper/2104.07645/) (2021)
- [Robust Imitation of Diverse Behaviors](https://mlsanity.com/paper/1707.02747/) (2017)
- [Learning human behaviors from motion capture by adversarial imitation](https://mlsanity.com/paper/1707.02201/) (2017)
- [Deep Object-Centric Representations for Generalizable Robot Learning](https://mlsanity.com/paper/1708.04225/) (2017)
- [Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery](https://mlsanity.com/paper/2105.01047/) (2021)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [Learning Contact-Rich Manipulation Skills with Guided Policy Search](https://mlsanity.com/paper/1501.05611/) (2015)
- [Visual Identification of Articulated Object Parts](https://mlsanity.com/paper/2012.00284/) (2020)
- [More than a Million Ways to Be Pushed: A High-Fidelity Experimental Dataset of Planar Pushing](https://mlsanity.com/paper/1604.04038/) (2016)
- [Generative Adversarial Self-Imitation Learning](https://mlsanity.com/paper/1812.00950/) (2018)
- [Robust and Adaptive Door Operation with a Mobile Robot](https://mlsanity.com/paper/1902.09051/) (2019)
- [Learning Hybrid Object Kinematics for Efficient Hierarchical Planning Under Uncertainty](https://mlsanity.com/paper/1907.09014/) (2019)
- [Improved Training with Curriculum GANs](https://mlsanity.com/paper/1807.09295/) (2018)

## Built on by (most cited first, 15 of 15)

- [UniDexGrasp: Universal Robotic Dexterous Grasping via Learning Diverse Proposal Generation and Goal-Conditioned Policy](https://mlsanity.com/paper/2303.00938/) (2023)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [3D-Aware Object Goal Navigation via Simultaneous Exploration and Identification](https://mlsanity.com/paper/2212.00338/) (2022)
- [PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations](https://mlsanity.com/paper/2303.16958/) (2023)
- [Improving Policy Optimization with Generalist-Specialist Learning](https://mlsanity.com/paper/2206.12984/) (2022)
- [PhysReaction: Physically Plausible Real-Time Humanoid Reaction Synthesis via Forward Dynamics Guided 4D Imitation](https://mlsanity.com/paper/2404.01081/) (2024)
- [SAGE: Bridging Semantic and Actionable Parts for GEneralizable Manipulation of Articulated Objects](https://mlsanity.com/paper/2312.01307/) (2023)
- [RPMArt: Towards Robust Perception and Manipulation for Articulated Objects](https://mlsanity.com/paper/2403.16023/) (2024)
- [Structure from Action: Learning Interactions for Articulated Object 3D Structure Discovery](https://mlsanity.com/paper/2207.08997/) (2022)
- [DeepIPC: Deeply Integrated Perception and Control for an Autonomous Vehicle in Real Environments](https://mlsanity.com/paper/2207.09934/) (2022)
- [Combining Planning and Diffusion for Mobility with Unknown Dynamics](https://mlsanity.com/paper/2410.06911/) (2024)
- [Decoupling Skill Learning from Robotic Control for Generalizable Object Manipulation](https://mlsanity.com/paper/2303.04016/) (2023)
- [Goal-based Self-Adaptive Generative Adversarial Imitation Learning (Goal-SAGAIL) for Multi-goal Robotic Manipulation Tasks](https://mlsanity.com/paper/2506.12676/) (2025)
- [Boosting Reinforcement Learning and Planning with Demonstrations: A Survey](https://mlsanity.com/paper/2303.13489/) (2023)
- [Learning the Generalizable Manipulation Skills on Soft-body Tasks via Guided Self-attention Behavior Cloning Policy](https://mlsanity.com/paper/2410.05756/) (2024)

## Abstract (arXiv)

Generalizable object manipulation skills are critical for intelligent and multi-functional robots to work in real-world complex scenes. Despite the recent progress in reinforcement learning, it is still very challenging to learn a generalizable manipulation policy that can handle a category of geometrically diverse articulated objects. In this work, we tackle this category-level object manipulation policy learning problem via imitation learning in a task-agnostic manner, where we assume no handcrafted dense rewards but only a terminal reward. Given this novel and challenging generalizable policy learning problem, we identify several key issues that can fail the previous imitation learning algorithms and hinder the generalization to unseen instances. We then propose several general but critical techniques, including generative adversarial self-imitation learning from demonstrations, progressive growing of discriminator, and instance-balancing for expert buffer, that accurately pinpoints and tackles these issues and can benefit category-level manipulation policy learning regardless of the tasks. Our experiments on ManiSkill benchmarks demonstrate a remarkable improvement on all tasks and our ablation studies further validate the contribution of each proposed technique.