# FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects

arXiv: https://arxiv.org/abs/2205.04382 · HTML: https://mlsanity.com/paper/2205.04382/

## Key facts

- Cited by 91 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 54 references that are in the map.
- First posted on arXiv 9 May 2022.

## Builds on (most cited first, 26 of 26)

- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Shape2Motion: Joint Analysis of Motion Parts and Attributes from 3D Shapes](https://mlsanity.com/paper/1903.03911/) (2019)
- [Reducing the Barrier to Entry of Complex Robotic Software: a MoveIt! Case Study](https://mlsanity.com/paper/1404.3785/) (2014)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [Deep Part Induction from Articulated Object Pairs](https://mlsanity.com/paper/1809.07417/) (2018)
- [UMPNet: Universal Manipulation Policy Network for Articulated Objects](https://mlsanity.com/paper/2109.05668/) (2021)
- [FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy](https://mlsanity.com/paper/2111.05623/) (2021)
- [Tactile-RL for Insertion: Generalization to Objects of Unknown Geometry](https://mlsanity.com/paper/2104.01167/) (2021)
- [Personalization of End-to-end Speech Recognition On Mobile Devices For Named Entities](https://mlsanity.com/paper/1912.09251/) (2019)
- [Planar Robot Casting with Real2Sim2Real Self-Supervised Learning](https://mlsanity.com/paper/2111.04814/) (2021)
- [Visual Identification of Articulated Object Parts](https://mlsanity.com/paper/2012.00284/) (2020)
- [RPM-Net: Recurrent Prediction of Motion and Parts from Point Cloud](https://mlsanity.com/paper/2006.14865/) (2020)
- [Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cables](https://mlsanity.com/paper/2011.04840/) (2020)
- [Learning Articulated Motions From Visual Demonstration](https://mlsanity.com/paper/1502.01659/) (2015)
- [Orienting Novel 3D Objects Using Self-Supervised Learning of Rotation Transforms](https://mlsanity.com/paper/2105.14246/) (2021)
- [Dex-Net 3.0: Computing Robust Robot Vacuum Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning](https://mlsanity.com/paper/1709.06670/) (2017)
- [Motion Perception in Reinforcement Learning with Dynamic Objects](https://mlsanity.com/paper/1901.03162/) (2019)
- [6-DoF Grasp Planning using Fast 3D Reconstruction and Grasp Quality CNN](https://mlsanity.com/paper/2009.08618/) (2020)

## Built on by (most cited first, 30 of 91)

- [RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation](https://mlsanity.com/paper/2412.13877/) (2024)
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [Flow as the Cross-Domain Manipulation Interface](https://mlsanity.com/paper/2407.15208/) (2024)
- [RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation](https://mlsanity.com/paper/2406.04339/) (2024)
- [General Flow as Foundation Affordance for Scalable Robot Learning](https://mlsanity.com/paper/2401.11439/) (2024)
- [PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation](https://mlsanity.com/paper/2601.03782/) (2026)
- [SplatSim: Zero-Shot Sim2Real Transfer of RGB Manipulation Policies Using Gaussian Splatting](https://mlsanity.com/paper/2409.10161/) (2024)
- [RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis](https://mlsanity.com/paper/2402.16117/) (2024)
- [End-to-End Affordance Learning for Robotic Manipulation](https://mlsanity.com/paper/2209.12941/) (2022)
- [PolarNet: 3D Point Clouds for Language-Guided Robotic Manipulation](https://mlsanity.com/paper/2309.15596/) (2023)
- [RoboGround: Robotic Manipulation with Grounded Vision-Language Priors](https://mlsanity.com/paper/2504.21530/) (2025)
- [TAX-Pose: Task-Specific Cross-Pose Estimation for Robot Manipulation](https://mlsanity.com/paper/2211.09325/) (2022)
- [Lift3D Foundation Policy: Lifting 2D Large-Scale Pretrained Models for Robust 3D Robotic Manipulation](https://mlsanity.com/paper/2411.18623/) (2024)
- [SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation](https://mlsanity.com/paper/2502.13143/) (2025)
- [PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations](https://mlsanity.com/paper/2303.16958/) (2023)
- [NovaFlow: Zero-Shot Manipulation via Actionable Flow from Generated Videos](https://mlsanity.com/paper/2510.08568/) (2025)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [A3VLM: Actionable Articulation-Aware Vision Language Model](https://mlsanity.com/paper/2406.07549/) (2024)
- [Frame Mining: a Free Lunch for Learning Robotic Manipulation from 3D Point Clouds](https://mlsanity.com/paper/2210.07442/) (2022)
- [NAP: Neural 3D Articulation Prior](https://mlsanity.com/paper/2305.16315/) (2023)
- [ManipVQA: Injecting Robotic Affordance and Physically Grounded Information into Multi-Modal Large Language Models](https://mlsanity.com/paper/2403.11289/) (2024)
- [DreamArt: Generating Interactable Articulated Objects from a Single Image](https://mlsanity.com/paper/2507.05763/) (2025)
- [Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flow](https://mlsanity.com/paper/2512.24766/) (2025)
- [DiffCLIP: Leveraging Stable Diffusion for Language Grounded 3D Classification](https://mlsanity.com/paper/2305.15957/) (2023)
- [Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions](https://mlsanity.com/paper/2309.07510/) (2023)
- [Challenges for Monocular 6D Object Pose Estimation in Robotics](https://mlsanity.com/paper/2307.12172/) (2023)
- [A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation](https://mlsanity.com/paper/2405.17418/) (2024)
- [SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulation](https://mlsanity.com/paper/2501.14400/) (2025)
- [3D Implicit Transporter for Temporally Consistent Keypoint Discovery](https://mlsanity.com/paper/2309.05098/) (2023)
- [AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation](https://mlsanity.com/paper/2406.11548/) (2024)

## Abstract (arXiv)

We explore a novel method to perceive and manipulate 3D articulated objects that generalizes to enable a robot to articulate unseen classes of objects. We propose a vision-based system that learns to predict the potential motions of the parts of a variety of articulated objects to guide downstream motion planning of the system to articulate the objects. To predict the object motions, we train a neural network to output a dense vector field representing the point-wise motion direction of the points in the point cloud under articulation. We then deploy an analytical motion planner based on this vector field to achieve a policy that yields maximum articulation. We train the vision system entirely in simulation, and we demonstrate the capability of our system to generalize to unseen object instances and novel categories in both simulation and the real world, deploying our policy on a Sawyer robot with no finetuning. Results show that our system achieves state-of-the-art performance in both simulated and real-world experiments.