# ArtReg: Visuo-Tactile based Pose Tracking and Manipulation of Unseen Articulated Objects

arXiv: https://arxiv.org/abs/2511.06378 · HTML: https://mlsanity.com/paper/2511.06378/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 50 references that are in the map.
- First posted on arXiv 9 Nov 2025.

## Builds on (most cited first, 24 of 24)

- [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017)
- [Open3D: A Modern Library for 3D Data Processing](https://mlsanity.com/paper/1801.09847/) (2018)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [Where2Act: From Pixels to Actions for Articulated 3D Objects](https://mlsanity.com/paper/2101.02692/) (2021)
- [3D Hand Shape and Pose Estimation from a Single RGB Image](https://mlsanity.com/paper/1903.00812/) (2019)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [Articulate-Anything: Automatic Modeling of Articulated Objects via a Vision-Language Foundation Model](https://mlsanity.com/paper/2410.13882/) (2024)
- [CAPTRA: CAtegory-level Pose Tracking for Rigid and Articulated Objects from Point Clouds](https://mlsanity.com/paper/2104.03437/) (2021)
- [UMPNet: Universal Manipulation Policy Network for Articulated Objects](https://mlsanity.com/paper/2109.05668/) (2021)
- [Act the Part: Learning Interaction Strategies for Articulated Object Part Discovery](https://mlsanity.com/paper/2105.01047/) (2021)
- [Articulated Object Interaction in Unknown Scenes with Whole-Body Mobile Manipulation](https://mlsanity.com/paper/2103.10534/) (2021)
- [FilterReg: Robust and Efficient Probabilistic Point-Set Registration using Gaussian Filter and Twist Parameterization](https://mlsanity.com/paper/1811.10136/) (2018)
- [A Probabilistic Framework for Learning Kinematic Models of Articulated Objects](https://mlsanity.com/paper/1405.7705/) (2014)
- [Towards Real-World Category-level Articulation Pose Estimation](https://mlsanity.com/paper/2105.03260/) (2021)
- [Understanding 3D Object Articulation in Internet Videos](https://mlsanity.com/paper/2203.16531/) (2022)
- [FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projection](https://mlsanity.com/paper/2306.12893/) (2023)
- [Category-Independent Articulated Object Tracking with Factor Graphs](https://mlsanity.com/paper/2205.03721/) (2022)
- [Learning Agent-Aware Affordances for Closed-Loop Interaction with Articulated Objects](https://mlsanity.com/paper/2209.05802/) (2022)
- [Let's Push Things Forward: A Survey on Robot Pushing](https://mlsanity.com/paper/1905.05138/) (2019)
- [Probabilistic Articulated Real-Time Tracking for Robot Manipulation](https://mlsanity.com/paper/1610.04871/) (2016)
- [Part-Guided 3D RL for Sim2Real Articulated Object Manipulation](https://mlsanity.com/paper/2404.17302/) (2024)
- [Goal-Driven Robotic Pushing Using Tactile and Proprioceptive Feedback](https://mlsanity.com/paper/2012.01859/) (2020)
- [A Code for Unscented Kalman Filtering on Manifolds (UKF-M)](https://mlsanity.com/paper/2002.00878/) (2020)
- [Learning Goal-Oriented Non-Prehensile Pushing in Cluttered Scenes](https://mlsanity.com/paper/2203.02389/) (2022)

## Abstract (arXiv)

Robots operating in real-world environments frequently encounter unknown objects with complex structures and articulated components, such as doors, drawers, cabinets, and tools. The ability to perceive, track, and manipulate these objects without prior knowledge of their geometry or kinematic properties remains a fundamental challenge in robotics. In this work, we present a novel method for visuo-tactile-based tracking of unseen objects (single, multiple, or articulated) during robotic interaction without assuming any prior knowledge regarding object shape or dynamics. Our novel pose tracking approach termed ArtReg (stands for Articulated Registration) integrates visuo-tactile point clouds in an unscented Kalman Filter formulation in the SE(3) Lie Group for point cloud registration. ArtReg is used to detect possible articulated joints in objects using purposeful manipulation maneuvers such as pushing or hold-pulling with a two-robot team. Furthermore, we leverage ArtReg to develop a closed-loop controller for goal-driven manipulation of articulated objects to move the object into the desired pose configuration. We have extensively evaluated our approach on various types of unknown objects through real robot experiments. We also demonstrate the robustness of our method by evaluating objects with varying center of mass, low-light conditions, and with challenging visual backgrounds. Furthermore, we benchmarked our approach on a standard dataset of articulated objects and demonstrated improved performance in terms of pose accuracy compared to state-of-the-art methods. Our experiments indicate that robust and accurate pose tracking leveraging visuo-tactile information enables robots to perceive and interact with unseen complex articulated objects (with revolute or prismatic joints).