# UMPNet: Universal Manipulation Policy Network for Articulated Objects

arXiv: https://arxiv.org/abs/2109.05668 · HTML: https://mlsanity.com/paper/2109.05668/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 37 references that are in the map.
- First posted on arXiv 13 Sep 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [Time-Contrastive Networks: Self-Supervised Learning from Video](https://mlsanity.com/paper/1704.06888/) (2017)
- [Learning Latent Plans from Play](https://mlsanity.com/paper/1903.01973/) (2019)
- [Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation](https://mlsanity.com/paper/1710.04615/) (2017)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Learning to Poke by Poking: Experiential Learning of Intuitive Physics](https://mlsanity.com/paper/1606.07419/) (2016)
- [Planning with Goal-Conditioned Policies](https://mlsanity.com/paper/1911.08453/) (2019)
- [ContactDB: Analyzing and Predicting Grasp Contact via Thermal Imaging](https://mlsanity.com/paper/1904.06830/) (2019)
- [TossingBot: Learning to Throw Arbitrary Objects with Residual Physics](https://mlsanity.com/paper/1903.11239/) (2019)
- [Grounded Human-Object Interaction Hotspots from Video](https://mlsanity.com/paper/1812.04558/) (2018)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [EGO-TOPO: Environment Affordances from Egocentric Video](https://mlsanity.com/paper/2001.04583/) (2020)
- [Grasping in the Wild:Learning 6DoF Closed-Loop Grasping from Low-Cost Demonstrations](https://mlsanity.com/paper/1912.04344/) (2019)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [Neural Task Graphs: Generalizing to Unseen Tasks from a Single Video Demonstration](https://mlsanity.com/paper/1807.03480/) (2018)
- [Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery](https://mlsanity.com/paper/2105.01047/) (2021)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [Learning Hybrid Object Kinematics for Efficient Hierarchical Planning Under Uncertainty](https://mlsanity.com/paper/1907.09014/) (2019)
- [Motion Reasoning for Goal-Based Imitation Learning](https://mlsanity.com/paper/1911.05864/) (2019)

## Built on by (most cited first, 30 of 65)

- [Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulation](https://mlsanity.com/paper/2209.05451/) (2022)
- [ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation](https://mlsanity.com/paper/2312.16217/) (2023)
- [UniDexGrasp++: Improving Dexterous Grasping Policy Learning via Geometry-aware Curriculum and Iterative Generalist-Specialist Learning](https://mlsanity.com/paper/2304.00464/) (2023)
- [RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulation](https://mlsanity.com/paper/2406.04339/) (2024)
- [GNFactor: Multi-Task Real Robot Learning with Generalizable Neural Feature Fields](https://mlsanity.com/paper/2308.16891/) (2023)
- [FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects](https://mlsanity.com/paper/2205.04382/) (2022)
- [DexArt: Benchmarking Generalizable Dexterous Manipulation with Articulated Objects](https://mlsanity.com/paper/2305.05706/) (2023)
- [RoboCodeX: Multimodal Code Generation for Robotic Behavior Synthesis](https://mlsanity.com/paper/2402.16117/) (2024)
- [AdaAfford: Learning to Adapt Manipulation Affordance for 3D Articulated Objects via Few-shot Interactions](https://mlsanity.com/paper/2112.00246/) (2021)
- [End-to-End Affordance Learning for Robotic Manipulation](https://mlsanity.com/paper/2209.12941/) (2022)
- [SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation](https://mlsanity.com/paper/2502.13143/) (2025)
- [PartManip: Learning Cross-Category Generalizable Part Manipulation Policy from Point Cloud Observations](https://mlsanity.com/paper/2303.16958/) (2023)
- [Where2Explore: Few-shot Affordance Learning for Unseen Novel Categories of Articulated Objects](https://mlsanity.com/paper/2309.07473/) (2023)
- [A3VLM: Actionable Articulation-Aware Vision Language Model](https://mlsanity.com/paper/2406.07549/) (2024)
- [HACMan: Learning Hybrid Actor-Critic Maps for 6D Non-Prehensile Manipulation](https://mlsanity.com/paper/2305.03942/) (2023)
- [K-VIL: Keypoints-based Visual Imitation Learning](https://mlsanity.com/paper/2209.03277/) (2022)
- [PhysPart: Physically Plausible Part Completion for Interactable Objects](https://mlsanity.com/paper/2408.13724/) (2024)
- [Category-Independent Articulated Object Tracking with Factor Graphs](https://mlsanity.com/paper/2205.03721/) (2022)
- [Ag2Manip: Learning Novel Manipulation Skills with Agent-Agnostic Visual and Action Representations](https://mlsanity.com/paper/2404.17521/) (2024)
- [Learning Environment-Aware Affordance for 3D Articulated Object Manipulation under Occlusions](https://mlsanity.com/paper/2309.07510/) (2023)
- [DNAct: Diffusion Guided Multi-Task 3D Policy Learning](https://mlsanity.com/paper/2403.04115/) (2024)
- [A Self-Correcting Vision-Language-Action Model for Fast and Slow System Manipulation](https://mlsanity.com/paper/2405.17418/) (2024)
- [3D Implicit Transporter for Temporally Consistent Keypoint Discovery](https://mlsanity.com/paper/2309.05098/) (2023)
- [AIC MLLM: Autonomous Interactive Correction MLLM for Robust Robotic Manipulation](https://mlsanity.com/paper/2406.11548/) (2024)
- [Composable Part-Based Manipulation](https://mlsanity.com/paper/2405.05876/) (2024)
- [Survey on Modeling of Human-made Articulated Objects](https://mlsanity.com/paper/2403.14937/) (2024)
- [Open-World Reinforcement Learning over Long Short-Term Imagination](https://mlsanity.com/paper/2410.03618/) (2024)
- [AdaManip: Adaptive Articulated Object Manipulation Environments and Policy Learning](https://mlsanity.com/paper/2502.11124/) (2025)
- [ArticuBot: Learning Universal Articulated Object Manipulation Policy via Large Scale Simulation](https://mlsanity.com/paper/2503.03045/) (2025)
- [Articulated 3D Scene Graphs for Open-World Mobile Manipulation](https://mlsanity.com/paper/2602.16356/) (2026)

## Abstract (arXiv)

We introduce the Universal Manipulation Policy Network (UMPNet) -- a single image-based policy network that infers closed-loop action sequences for manipulating arbitrary articulated objects. To infer a wide range of action trajectories, the policy supports 6DoF action representation and varying trajectory length. To handle a diverse set of objects, the policy learns from objects with different articulation structures and generalizes to unseen objects or categories. The policy is trained with self-guided exploration without any human demonstrations, scripted policy, or pre-defined goal conditions. To support effective multi-step interaction, we introduce a novel Arrow-of-Time action attribute that indicates whether an action will change the object state back to the past or forward into the future. With the Arrow-of-Time inference at each interaction step, the learned policy is able to select actions that consistently lead towards or away from a given state, thereby, enabling both effective state exploration and goal-conditioned manipulation. Video is available at https://youtu.be/KqlvcL9RqKM