mlsanityData to 7 Oct 2026

Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery

arXiv 2105.01047 ↗First posted 3 May 2021cs.CVcited by 45

Not on the map. Why?

Abstract (arXiv)

People often use physical intuition when manipulating articulated objects, irrespective of object semantics. Motivated by this observation, we identify an important embodied task where an agent must play with objects to recover their parts. To this end, we introduce Act the Part (AtP) to learn how to interact with articulated objects to discover and segment their pieces. By coupling action selection and motion segmentation, AtP is able to isolate structures to make perceptual part recovery possible without semantic labels. Our experiments show AtP learns efficient strategies for part discovery, can generalize to unseen categories, and is capable of conditional reasoning for the task. Although trained in simulation, we show convincing transfer to real world data with no fine-tuning.

Key facts. Cited by 45 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 3 May 2021.

Builds on

35 of 56 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016PointNet: Deep Learning on Point Sets for 3D Classification and Segmentationcited by 5,223
  6. 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,384
  7. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  8. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  9. 2020SAPIEN: A SimulAted Part-based Interactive ENvironmentcited by 584
  10. 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 521
  11. 2016YouTube-8M: A Large-Scale Video Classification Benchmarkcited by 324
  12. 2021Where2Act: From Pixels to Actions for Articulated 3D Objectscited by 179
23 more
  1. 2019Category-Level Articulated Object Pose Estimationcited by 158
  2. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 121
  3. 2018Unsupervised Learning of Object Landmarks through Conditional Image Generationcited by 113
  4. 2019SCOPS: Self-Supervised Co-Part Segmentationcited by 100
  5. 2016Interactive Perception: Leveraging Action in Perception and Perception in Actioncited by 77
  6. 2018Deep Part Induction from Articulated Object Pairscited by 66
  7. 2019DensePhysNet: Learning Dense Physical Object Representations via Multi-step Dynamic Interactionscited by 61
  8. 2016SE3-Nets: Learning Rigid Body Motion using Deep Neural Networkscited by 48
  9. 2019Camera-to-Robot Pose Estimation from a Single Imagecited by 48
  10. 2021MultiBodySync: Multi-Body Segmentation and Motion Estimation via 3D Scan Synchronizationcited by 46
  11. 2014A Probabilistic Framework for Learning Kinematic Models of Articulated Objectscited by 40
  12. 2020Learning 3D Dynamic Scene Representations for Robot Manipulationcited by 39
  13. 2016The Curious Robot: Learning Visual Representations via Physical Interactionscited by 37
  14. 2020Nothing But Geometric Constraints: A Model-Free Method for Articulated Object Pose Estimationcited by 25
  15. 2014Semantic Part Segmentation using Compositional Model combining Shape and Appearancecited by 22
  16. 20203D-OES: Viewpoint-Invariant Object-Factorized Environment Simulatorscited by 21
  17. 2019Unsupervised Discovery of Parts, Structure, and Dynamicscited by 20
  18. 2018Learning Instance Segmentation by Interactioncited by 18
  19. 2015Learning Articulated Motions From Visual Demonstrationcited by 17
  20. 2020Learning About Objects by Learning to Interact with Themcited by 14
  21. 2020Self-supervised Transfer Learning for Instance Segmentation through Physical Interactioncited by 13
  22. 2020Hindsight for Foresight: Unsupervised Structured Dynamics Models from Physical Interactioncited by 8
  23. 2015Deep Learning for Semantic Part Segmentation with High-Level Guidancecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 8; 2023: 10; 2024: 6; 2025: 12; 2026: 3.

45 papers.

Sort:

By area:

  1. 2022Ditto: Building Digital Twins of Articulated Objects from Interactioncited by 152
  2. 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 135
  3. 2023PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objectscited by 115
  4. 2021VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objectscited by 78
  5. 2025ArtGS: Building Interactable Replicas of Complex Articulated Objects via Gaussian Splattingcited by 67
  6. 2021UMPNet: Universal Manipulation Policy Network for Articulated Objectscited by 64
  7. 2023DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objectscited by 64
  8. 2024Real2Code: Reconstruct Articulated Objects via Code Generationcited by 64
  9. 2021Unsupervised Part Discovery from Contrastive Reconstructioncited by 57
  10. 20223D-Aware Object Goal Navigation via Simultaneous Exploration and Identificationcited by 46
  11. 2025Articulate AnyMesh: Open-Vocabulary 3D Articulated Objects Modelingcited by 41
  12. 2023Ditto in the House: Building Articulation Models of Indoor Scenes through Interactive Perceptioncited by 40
33 more
  1. 2022Continuous Scene Representations for Embodied AIcited by 39
  2. 2023Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objectscited by 33
  3. 2024DragAPart: Learning a Part-Level Motion Prior for Articulated Objectscited by 32
  4. 2025EmbodiedGen: Towards a Generative 3D World Engine for Embodied Intelligencecited by 32
  5. 2023Learning Foresightful Dense Visual Affordance for Deformable Object Manipulationcited by 25
  6. 2023Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusionscited by 23
  7. 2025DIPO: Dual-State Images Controlled Articulated Object Generation Powered by Diverse Datacited by 21
  8. 2022DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulationcited by 18
  9. 2022Learning Category-Level Generalizable Object Manipulation Policy via Generative Adversarial Self-Imitation Learning from Demonstrationscited by 15
  10. 2025VideoArtGS: Building Digital Twins of Articulated Objects from Monocular Videocited by 15
  11. 2024Survey on Modeling of Human-made Articulated Objectscited by 14
  12. 2025SplArt: Articulation Estimation and Part-Level Reconstruction with 3D Gaussian Splattingcited by 13
  13. 2023STRAP: Structured Object Affordance Segmentation with Point Supervisioncited by 10
  14. 2021Robot Skill Adaptation via Soft Actor-Critic Gaussian Mixture Modelscited by 9
  15. 2021IFR-Explore: Learning Inter-object Functional Relationships in 3D Indoor Scenescited by 9
  16. 2025GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulationcited by 9
  17. 2025ScrewSplat: An End-to-End Method for Articulated Object Recognitioncited by 9
  18. 2026AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understandingcited by 8
  19. 2021Scene Editing as Teleoperation: A Case Study in 6DoF Kit Assemblycited by 6
  20. 2023Learning Part Motion of Articulated Objects Using Spatially Continuous Neural Implicit Representationscited by 5
  21. 2023SAGE: Bridging Semantic and Actionable Parts for GEneralizable Manipulation of Articulated Objectscited by 5
  22. 2024PAPR in Motion: Seamless Point-level 3D Scene Interpolationcited by 5
  23. 2022Structure from Action: Learning Interactions for Articulated Object 3D Structure Discoverycited by 4
  24. 2023Learning Part Segmentation from Synthetic Animalscited by 4
  25. 2022BusyBot: Learning to Interact, Reason, and Plan in a BusyBoard Environmentcited by 3
  26. 2024Articulated Object Manipulation using Online Axis Estimation with SAM2-Based Trackingcited by 3
  27. 2025You Only Estimate Once: Unified, One-stage, Real-Time Category-level Articulated Object 6D Pose Estimation for Robotic Graspingcited by 3
  28. 2024Learning to Look: Seeking Information for Decision Making via Policy Factorizationcited by 2
  29. 2025TopoCut: Learning Multi-Step Cutting with Spectral Rewards and Discrete Diffusion Policiescited by 2
  30. 2025Part2GS: Part-aware Modeling of Articulated Objects using 3D Gaussian Splattingcited by 1
  31. 2025ArtReg: Visuo-Tactile based Pose Tracking and Manipulation of Unseen Articulated Objectscited by 0
  32. 2026Test-Time Adaptation for Tactile-Vision-Language Modelscited by 0
  33. 2026FAMOS: Feed-Forward 3D Articulation Modeling from Sparse Observationscited by 0