mlsanityData to 14 Oct 2026

GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Parts

arXiv 2211.05272 ↗First posted 10 Nov 2022cs.CVcited by 139

Not on the map. Why?

Abstract (arXiv)

For years, researchers have been devoted to generalizable object perception and manipulation, where cross-category generalizability is highly desired yet underexplored. In this work, we propose to learn such cross-category skills via Generalizable and Actionable Parts (GAParts). By identifying and defining 9 GAPart classes (lids, handles, etc.) in 27 object categories, we construct a large-scale part-centric interactive dataset, GAPartNet, where we provide rich, part-level annotations (semantics, poses) for 8,489 part instances on 1,166 objects. Based on GAPartNet, we investigate three cross-category tasks: part segmentation, part pose estimation, and part-based object manipulation. Given the significant domain gaps between seen and unseen object categories, we propose a robust 3D segmentation method from the perspective of domain generalization by integrating adversarial learning techniques. Our method outperforms all existing methods by a large margin, no matter on seen or unseen categories. Furthermore, with part segmentation and pose estimation results, we leverage the GAPart pose definition to design part-based manipulation heuristics that can generalize well to unseen object categories in both the simulator and the real world. Our dataset, code, and demos are available on our project page.

Key facts. Cited by 139 papers in the map (data through 14 Oct 2026). Builds on 62 of its 76 references that are in the map. First posted on arXiv 10 Nov 2022.

Builds on

62 of 76 references.

References · 62

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,639
  3. 2017Mask R-CNNcited by 7,277
  4. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,248
  5. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,044
  8. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  9. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,223
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  11. 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,331
  12. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 775
38 more
  1. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
  2. 2018PointCNN: Convolution On X-Transformed Pointscited by 764
  3. 2017Learning to Generalize: Meta-Learning for Domain Generalizationcited by 763
  4. 2013Domain Generalization via Invariant Feature Representationcited by 623
  5. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 595
  6. 2021Domain Generalization with MixStylecited by 566
  7. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
  8. 2019Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimationcited by 495
  9. 2019PointRend: Image Segmentation as Renderingcited by 440
  10. 2018PVNet: Pixel-wise Voting Network for 6DoF Pose Estimationcited by 385
  11. 2019DenseFusion: 6D Object Pose Estimation by Iterative Dense Fusioncited by 355
  12. 2020PointGroup: Dual-Set Point Grouping for 3D Instance Segmentationcited by 321
  13. 2020Learning to Generate Novel Domains for Domain Generalizationcited by 296
  14. 2020CosyPose: Consistent multi-view multi-object 6D pose estimationcited by 287
  15. 2018DeepIM: Deep Iterative Matching for 6D Pose Estimationcited by 283
  16. 2019Episodic Training for Domain Generalizationcited by 245
  17. 2017Submanifold Sparse Convolutional Networkscited by 223
  18. 2022SoftGroup for 3D Instance Segmentation on Point Cloudscited by 200
  19. 2021ManiSkill: Generalizable Manipulation Skill Benchmark with Large-Scale Demonstrationscited by 197
  20. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 182
  21. 2019Implicit 3D Orientation Learning for 6D Object Detection from RGB Imagescited by 170
  22. 2017SGPN: Similarity Group Proposal Network for 3D Point Cloud Instance Segmentationcited by 169
  23. 2020Shape Prior Deformation for Categorical 6D Object Pose and Size Estimationcited by 161
  24. 2019Category-Level Articulated Object Pose Estimationcited by 159
  25. 2018GSPN: Generative Shape Proposal Network for 3D Instance Segmentation in Point Cloudcited by 146
  26. 2021FFB6D: A Full Flow Bidirectional Fusion Network for 6D Pose Estimationcited by 144
  27. 2021Hierarchical Aggregation for 3D Instance Segmentationcited by 127
  28. 2020Learning Canonical Shape Space for Category-Level 6D Object Pose and Size Estimationcited by 124
  29. 2021FS-Net: Fast Shape-based Network for Category-Level 6D Object Pose Estimation with Decoupled Rotation Mechanismcited by 121
  30. 2019Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapescited by 119
  31. 2021Synergies Between Affordance and Geometry: 6-DoF Grasp Detection via Implicit Representationscited by 108
  32. 2021Neural Parts: Learning Expressive 3D Shape Abstractions with Invertible Neural Networkscited by 103
  33. 2021Volumetric Grasping Network: Real-time 6 DOF Grasp Detection in Cluttercited by 95
  34. 2021CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Cloudscited by 84
  35. 2022AKB-48: A Real-World Articulated Object Knowledge Basecited by 82
  36. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 80
  37. 20196-PACK: Category-level 6D Pose Tracker with Anchor-Based Keypointscited by 78
  38. 2021Category-Level 6D Object Pose Estimation via Cascaded Relation and Recurrent Reconstruction Networkscited by 69

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 16; 2024: 44; 2025: 37; 2026: 42.

139 papers.

Sort:

By area:

  1. 2024Aligning Cyber Space with Physical World: A Comprehensive Survey on Embodied AIcited by 179
  2. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulationcited by 162
  3. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 137
  4. 2024PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AIcited by 125
  5. 2023UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learningcited by 107
  6. 2024Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Modelcited by 96
  7. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 85
  8. 2023PhysHOI: Physics-Based Imitation of Dynamic Human-Object Interactioncited by 76
  9. 2024Robo-ABC: Affordance Generalization Beyond Categories via Semantic Correspondence for Robot Manipulationcited by 71
  10. 2023ARNOLD: A Benchmark for Language-Grounded Task Learning With Continuous States in Realistic 3D Scenescited by 69
  11. 2024SINGAPO: Single Image Controlled Generation of Articulated Parts in Objectscited by 69
  12. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
38 more
  1. 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 66
  2. 2023Gen2Sim: Scaling up Robot Learning in Simulation with Generative Modelscited by 64
  3. 2023Grounding 3D Object Affordance from 2D Interactions in Imagescited by 62
  4. 2024RAM: Retrieval-Based Affordance Transfer for Generalizable Zero-Shot Robotic Manipulationcited by 58
  5. 2024Robot See Robot Do: Imitating Articulated Object Manipulation with Monocular 4D Reconstructioncited by 54
  6. 2025RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learningcited by 54
  7. 2025InternData-A1: Pioneering High-Fidelity Synthetic Data for Pre-training Generalist Policycited by 50
  8. 2023LEMON: Learning 3D Human-Object Interaction Relation from 2D Imagescited by 49
  9. 2025VidBot: Learning Generalizable 3D Actions from In-the-Wild 2D Human Videos for Zero-Shot Robotic Manipulationcited by 46
  10. 2025Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modelingcited by 43
  11. 2023PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observationscited by 42
  12. 2025PhysX-3D: Physical-Grounded 3D Asset Generationcited by 42
  13. 2024MeshArt: Generating Articulated Meshes with Structure-Guided Transformerscited by 41
  14. 2024AffordDP: Generalizable Diffusion Policy with Transferable Affordancecited by 37
  15. 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 35
  16. 2024A Survey of Embodied Learning for Object-Centric Robotic Manipulationcited by 35
  17. 2025ArticulatedGS: Self-supervised Digital Twin Modeling of Articulated Objects using 3D Gaussian Splattingcited by 35
  18. 2024A3VLM: Actionable Articulation-Aware Vision Language Modelcited by 33
  19. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  20. 2024SkillMimic: Learning Basketball Interaction Skills from Demonstrationscited by 30
  21. 2026Articraft: An Agentic System for Scalable Articulated 3D Asset Generationcited by 30
  22. 2024Puppet-Master: Scaling Interactive Video Generation as a Motion Prior for Part-Level Dynamicscited by 28
  23. 2024EgoChoir: Capturing 3D Human-Object Interaction Regions from Egocentric Viewscited by 26
  24. 2025PartNeXt: A Next-Generation Dataset for Fine-Grained and Hierarchical 3D Part Understandingcited by 26
  25. 2023NAP: Neural 3D Articulation Priorcited by 25
  26. 2024PhysPart: Physically Plausible Part Completion for Interactable Objectscited by 25
  27. 2025DreamArt: Generating Interactable Articulated Objects from a Single Imagecited by 25
  28. 2024Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representationscited by 24
  29. 2024Articulate your NeRF: Unsupervised articulated object modeling via conditional view synthesiscited by 24
  30. 2024UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondencecited by 23
  31. 2025Particulate: Feed-Forward 3D Object Articulationcited by 23
  32. 2024Learning 2D Invariant Affordance Knowledge for 3D Affordance Groundingcited by 22
  33. 2024S2O: Static to Openable Enhancement for Articulated 3D Objectscited by 22
  34. 2025DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Datacited by 21
  35. 2023Kinematic-aware Prompting for Generalizable Articulated Object Manipulation with LLMscited by 19
  36. 2025Grounding 3D Object Affordance with Language Instructions, Visual Observations and Interactionscited by 19
  37. 2023ZeroPS: High-quality Cross-modal Knowledge Transfer for Zero-Shot 3D Part Segmentationcited by 18
  38. 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 18

50 most cited of 139 All 139 (JSON)