# RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud

arXiv: https://arxiv.org/abs/2006.14865 · HTML: https://mlsanity.com/paper/2006.14865/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jun 2020.

## Built on by (most cited first, 26 of 26)

- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [PARIS: Part-level Reconstruction and Motion Analysis for Articulated Objects](https://mlsanity.com/paper/2308.07391/) (2023)
- [FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects](https://mlsanity.com/paper/2205.04382/) (2022)
- [VAT-Mart: Learning Visual Action Trajectory Proposals for Manipulating 3D ARTiculated Objects](https://mlsanity.com/paper/2106.14440/) (2021)
- [JoinABLe: Learning Bottom-up Assembly of Parametric CAD Joints](https://mlsanity.com/paper/2111.12772/) (2021)
- [AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions](https://mlsanity.com/paper/2112.00246/) (2021)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [Building Rearticulable Models for Arbitrary 3D Objects from 4D Point Clouds](https://mlsanity.com/paper/2306.00979/) (2023)
- [NAP: Neural 3D Articulation Prior](https://mlsanity.com/paper/2305.16315/) (2023)
- [S2O: Static to Openable Enhancement for Articulated 3D Objects](https://mlsanity.com/paper/2409.18896/) (2024)
- [Morig: Motion-aware rigging of character meshes from point clouds](https://mlsanity.com/paper/2210.09463/) (2022)
- [iTACO: Interactable Digital Twins of Articulated Objects from Casually Captured RGBD Videos](https://mlsanity.com/paper/2506.08334/) (2025)
- [Distributional Depth-Based Estimation of Object Articulation Models](https://mlsanity.com/paper/2108.05875/) (2021)
- [GaussianArt: Unified Modeling of Geometry and Motion for Articulated Objects](https://mlsanity.com/paper/2508.14891/) (2025)
- [Semi-Weakly Supervised Object Kinematic Motion Prediction](https://mlsanity.com/paper/2303.17774/) (2023)
- [Survey on Modeling of Human-made Articulated Objects](https://mlsanity.com/paper/2403.14937/) (2024)
- [Unsupervised Kinematic Motion Detection for Part-segmented 3D Shape Collections](https://mlsanity.com/paper/2206.08497/) (2022)
- [Learning to Infer Kinematic Hierarchies for Novel Object Instances](https://mlsanity.com/paper/2110.07911/) (2021)
- [NARF22: Neural Articulated Radiance Fields for Configuration-Aware Rendering](https://mlsanity.com/paper/2210.01166/) (2022)
- [Articulation in Motion: Prior-free Part Mobility Analysis for Articulated Objects By Dynamic-Static Disentanglement](https://mlsanity.com/paper/2603.02910/) (2026)
- [Active Coarse-to-Fine Segmentation of Moveable Parts from Real Images](https://mlsanity.com/paper/2303.11530/) (2023)
- [Interaction-Driven Active 3D Reconstruction with Object Interiors](https://mlsanity.com/paper/2310.14700/) (2023)
- [Shape Completion with Points in the Shadow](https://mlsanity.com/paper/2209.08345/) (2022)
- [CA2T-Net: Category-Agnostic 3D Articulation Transfer from Single Image](https://mlsanity.com/paper/2301.02232/) (2023)
- [Mates2Motion: Learning How Mechanical CAD Assemblies Work](https://mlsanity.com/paper/2208.01779/) (2022)
- [One-to-many Reconstruction of 3D Geometry of cultural Artifacts using a synthetically trained Generative Model](https://mlsanity.com/paper/2402.08310/) (2024)

## Abstract (arXiv)

We introduce RPM-Net, a deep learning-based approach which simultaneously infers movable parts and hallucinates their motions from a single, un-segmented, and possibly partial, 3D point cloud shape. RPM-Net is a novel Recurrent Neural Network (RNN), composed of an encoder-decoder pair with interleaved Long Short-Term Memory (LSTM) components, which together predict a temporal sequence of pointwise displacements for the input point cloud. At the same time, the displacements allow the network to learn movable parts, resulting in a motion-based shape segmentation. Recursive applications of RPM-Net on the obtained parts can predict finer-level part motions, resulting in a hierarchical object segmentation. Furthermore, we develop a separate network to estimate part mobilities, e.g., per-part motion parameters, from the segmented motion sequence. Both networks learn deep predictive models from a training set that exemplifies a variety of mobilities for diverse objects. We show results of simultaneous motion and part predictions from synthetic and real scans of 3D objects exhibiting a variety of part mobilities, possibly involving multiple movable parts.