# Tracking Partially-Occluded Deformable Objects while Enforcing Geometric Constraints

arXiv: https://arxiv.org/abs/2011.00627 · HTML: https://mlsanity.com/paper/2011.00627/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2020.

## Built on by (most cited first, 12 of 12)

- [Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation](https://mlsanity.com/paper/2206.02881/) (2022)
- [Understanding 3D Object Interaction from a Single Image](https://mlsanity.com/paper/2305.09664/) (2023)
- [Self-supervised Cloth Reconstruction via Action-conditioned Cloth Tracking](https://mlsanity.com/paper/2302.09502/) (2023)
- [Deformable One-Dimensional Object Detection for Routing and Manipulation](https://mlsanity.com/paper/2201.06775/) (2022)
- [Detection and Physical Interaction with Deformable Linear Objects](https://mlsanity.com/paper/2205.08041/) (2022)
- [Image Deformation Estimation via Multi-Objective Optimization](https://mlsanity.com/paper/2106.04139/) (2021)
- [DLOFTBs -- Fast Tracking of Deformable Linear Objects with B-splines](https://mlsanity.com/paper/2302.13694/) (2023)
- [RaggeDi: Diffusion-based State Estimation of Disordered Rags, Sheets, Towels and Blankets](https://mlsanity.com/paper/2409.11831/) (2024)
- [Implicit Contact Diffuser: Sequential Contact Reasoning with Latent Point Cloud Diffusion](https://mlsanity.com/paper/2410.16571/) (2024)
- [DLO-Splatting: Tracking Deformable Linear Objects Using 3D Gaussian Splatting](https://mlsanity.com/paper/2505.08644/) (2025)
- [KnotDLO: Toward Interpretable Knot Tying](https://mlsanity.com/paper/2506.22176/) (2025)
- [TrackDeform3D: Markerless and Autonomous 3D Keypoint Tracking and Dataset Collection for Deformable Objects](https://mlsanity.com/paper/2603.17068/) (2026)

## Abstract (arXiv)

In order to manipulate a deformable object, such as rope or cloth, in unstructured environments, robots need a way to estimate its current shape. However, tracking the shape of a deformable object can be challenging because of the object's high flexibility, (self-)occlusion, and interaction with obstacles. Building a high-fidelity physics simulation to aid in tracking is difficult for novel environments. Instead we focus on tracking the object based on RGBD images and geometric motion estimates and obstacles. Our key contributions over previous work in this vein are: 1) A better way to handle severe occlusion by using a motion model to regularize the tracking estimate; and 2) The formulation of textitconvex geometric constraints, which allow us to prevent self-intersection and penetration into known obstacles via a post-processing step. These contributions allow us to outperform previous methods by a large margin in terms of accuracy in scenarios with severe occlusion and obstacles.