# Online Estimation and Manipulation of Articulated Objects

arXiv: https://arxiv.org/abs/2601.01438 · HTML: https://mlsanity.com/paper/2601.01438/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 4 Jan 2026.

## Builds on (most cited first, 23 of 23)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Affordances from Human Videos as a Versatile Representation for Robotics](https://mlsanity.com/paper/2304.08488/) (2023)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Ditto: Building Digital Twins of Articulated Objects from Interaction](https://mlsanity.com/paper/2202.08227/) (2022)
- [FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects](https://mlsanity.com/paper/2205.04382/) (2022)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [UMPNet: Universal Manipulation Policy Network for Articulated Objects](https://mlsanity.com/paper/2109.05668/) (2021)
- [OPD: Single-view 3D Openable Part Detection](https://mlsanity.com/paper/2203.16421/) (2022)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [A Probabilistic Framework for Learning Kinematic Models of Articulated Objects](https://mlsanity.com/paper/1405.7705/) (2014)
- [Visual Identification of Articulated Object Parts](https://mlsanity.com/paper/2012.00284/) (2020)
- [FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projection](https://mlsanity.com/paper/2306.12893/) (2023)
- [Category-Independent Articulated Object Tracking with Factor Graphs](https://mlsanity.com/paper/2205.03721/) (2022)
- [Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects](https://mlsanity.com/paper/2209.05802/) (2022)
- [Online Estimation of Articulated Objects with Factor Graphs using Vision and Proprioceptive Sensing](https://mlsanity.com/paper/2309.16343/) (2023)
- [Kineverse: A Symbolic Articulation Model Framework for Model-Agnostic Mobile Manipulation](https://mlsanity.com/paper/2012.05362/) (2020)
- [Estimating 3D Motion and Forces of Human-Object Interactions from Internet Videos](https://mlsanity.com/paper/2111.01591/) (2021)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Few-Shot Learning of Force-Based Motions From Demonstration Through Pre-training of Haptic Representation](https://mlsanity.com/paper/2309.04640/) (2023)
- [Structure from Action: Learning Interactions for Articulated Object 3D Structure Discovery](https://mlsanity.com/paper/2207.08997/) (2022)

## Built on by (most cited first, 3 of 3)

- [DragMesh-2: Physically Plausible Dexterous Hand-Object Interaction with Articulated Objects](https://mlsanity.com/paper/2606.15133/) (2026)
- [From Perception to Assistance: Open-Vocabulary Shared Autonomy for Robotic Manipulation](https://mlsanity.com/paper/2607.17323/) (2026)
- [ForceTwin: Physics-informed Digital Twins for Robotic Manipulation from Instrumented Human Interaction](https://mlsanity.com/paper/2609.21751/) (2026)

## Abstract (arXiv)

From refrigerators to kitchen drawers, humans interact with articulated objects effortlessly every day while completing household chores. For automating these tasks, service robots must be capable of manipulating arbitrary articulated objects. Recent deep learning methods have been shown to predict valuable priors on the affordance of articulated objects from vision. In contrast, many other works estimate object articulations by observing the articulation motion, but this requires the robot to already be capable of manipulating the object. In this article, we propose a novel approach combining these methods by using a factor graph for online estimation of articulation which fuses learned visual priors and proprioceptive sensing during interaction into an analytical model of articulation based on Screw Theory. With our method, a robotic system makes an initial prediction of articulation from vision before touching the object, and then quickly updates the estimate from kinematic and force sensing during manipulation. We evaluate our method extensively in both simulations and real-world robotic manipulation experiments. We demonstrate several closed-loop estimation and manipulation experiments in which the robot was capable of opening previously unseen drawers. In real hardware experiments, the robot achieved a 75% success rate for autonomous opening of unknown articulated objects.