# Learning Sequential Kinematic Models from Demonstrations for Multi-Jointed Articulated Objects

arXiv: https://arxiv.org/abs/2505.06363 · HTML: https://mlsanity.com/paper/2505.06363/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 9 May 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [SAPIEN: A SimulAted Part-based Interactive ENvironment](https://mlsanity.com/paper/2003.08515/) (2020)
- [PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understanding](https://mlsanity.com/paper/1812.02713/) (2018)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objects](https://mlsanity.com/paper/2205.04382/) (2022)
- [ScrewNet: Category-Independent Articulation Model Estimation From Depth Images Using Screw Theory](https://mlsanity.com/paper/2008.10518/) (2020)
- [Deep Part Induction from Articulated Object Pairs](https://mlsanity.com/paper/1809.07417/) (2018)
- [A Probabilistic Framework for Learning Kinematic Models of Articulated Objects](https://mlsanity.com/paper/1405.7705/) (2014)
- [FlowBot++: Learning Generalized Articulated Objects Manipulation via Articulation Projection](https://mlsanity.com/paper/2306.12893/) (2023)
- [Distributional Depth-Based Estimation of Object Articulation Models](https://mlsanity.com/paper/2108.05875/) (2021)
- [Learning Articulated Motions From Visual Demonstration](https://mlsanity.com/paper/1502.01659/) (2015)

## Built on by (most cited first, 1 of 1)

- [Advances in 4D Representation: Geometry, Motion, and Interaction](https://mlsanity.com/paper/2510.19255/) (2025)

## Abstract (arXiv)

As robots become more generalized and deployed in diverse environments, they must interact with complex objects, many with multiple independent joints or degrees of freedom (DoF) requiring precise control. A common strategy is object modeling, where compact state-space models are learned from real-world observations and paired with classical planning. However, existing methods often rely on prior knowledge or focus on single-DoF objects, limiting their applicability. They also fail to handle occluded joints and ignore the manipulation sequences needed to access them. We address this by learning object models from human demonstrations. We introduce Object Kinematic Sequence Machines (OKSMs), a novel representation capturing both kinematic constraints and manipulation order for multi-DoF objects. To estimate these models from point cloud data, we present Pokenet, a deep neural network trained on human demonstrations. We validate our approach on 8,000 simulated and 1,600 real-world annotated samples. Pokenet improves joint axis and state estimation by over 20 percent on real-world data compared to prior methods. Finally, we demonstrate OKSMs on a Sawyer robot using inverse kinematics-based planning to manipulate multi-DoF objects.