# GarmentTracking: Category-Level Garment Pose Tracking

arXiv: https://arxiv.org/abs/2303.13913 · HTML: https://mlsanity.com/paper/2303.13913/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 44 references that are in the map.
- First posted on arXiv 24 Mar 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [3D U-Net: Learning Dense Volumetric Segmentation from Sparse Annotation](https://mlsanity.com/paper/1606.06650/) (2016)
- [Normalized Object Coordinate Space for Category-Level 6D Object Pose and Size Estimation](https://mlsanity.com/paper/1901.02970/) (2019)
- [Category-Level Articulated Object Pose Estimation](https://mlsanity.com/paper/1912.11913/) (2019)
- [4DComplete: Non-Rigid Motion Estimation Beyond the Observable Surface](https://mlsanity.com/paper/2105.01905/) (2021)
- [CLOTH3D: Clothed 3D Humans](https://mlsanity.com/paper/1912.02792/) (2019)
- [Deep Fashion3D: A Dataset and Benchmark for 3D Garment Reconstruction from Single Images](https://mlsanity.com/paper/2003.12753/) (2020)
- [NPMs: Neural Parametric Models for 3D Deformable Shapes](https://mlsanity.com/paper/2104.00702/) (2021)
- [AKB-48: A Real-World Articulated Object Knowledge Base](https://mlsanity.com/paper/2202.08432/) (2022)
- [FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding](https://mlsanity.com/paper/2105.03655/) (2021)
- [DeepDeform: Learning Non-rigid RGB-D Reconstruction with Semi-supervised Data](https://mlsanity.com/paper/1912.04302/) (2019)
- [VolumeDeform: Real-time Volumetric Non-rigid Reconstruction](https://mlsanity.com/paper/1603.08161/) (2016)
- [PTTR: Relational 3D Point Cloud Object Tracking with Transformer](https://mlsanity.com/paper/2112.02857/) (2021)
- [SpeedFolding: Learning Efficient Bimanual Folding of Garments](https://mlsanity.com/paper/2208.10552/) (2022)
- [Neural Non-Rigid Tracking](https://mlsanity.com/paper/2006.13240/) (2020)
- [GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completion](https://mlsanity.com/paper/2104.05177/) (2021)
- [Towards Real-World Category-level Articulation Pose Estimation](https://mlsanity.com/paper/2105.03260/) (2021)
- [SPAMs: Structured Implicit Parametric Models](https://mlsanity.com/paper/2201.08141/) (2022)
- [OcclusionFusion: Occlusion-aware Motion Estimation for Real-time Dynamic 3D Reconstruction](https://mlsanity.com/paper/2203.07977/) (2022)
- [Garment4D: Garment Reconstruction from Point Cloud Sequences](https://mlsanity.com/paper/2112.04159/) (2021)
- [OMAD: Object Model with Articulated Deformations for Pose Estimation and Retrieval](https://mlsanity.com/paper/2112.07334/) (2021)

## Built on by (most cited first, 16 of 16)

- [UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence](https://mlsanity.com/paper/2405.06903/) (2024)
- [GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation](https://mlsanity.com/paper/2411.01200/) (2024)
- [UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Folding](https://mlsanity.com/paper/2311.01267/) (2023)
- [Reconstruction of Manipulated Garment with Guided Deformation Prior](https://mlsanity.com/paper/2405.10934/) (2024)
- [TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach](https://mlsanity.com/paper/2407.03245/) (2024)
- [InstructionBench: An Instructional Video Understanding Benchmark](https://mlsanity.com/paper/2504.05040/) (2025)
- [CAP-Net: A Unified Network for 6D Pose and Size Estimation of Categorical Articulated Parts from a Single RGB-D Image](https://mlsanity.com/paper/2504.11230/) (2025)
- [BiFold: Bimanual Cloth Folding with Language Guidance](https://mlsanity.com/paper/2501.16458/) (2025)
- [GraphGarment: Learning Garment Dynamics for Bimanual Cloth Manipulation Tasks](https://mlsanity.com/paper/2503.05817/) (2025)
- [You Only Estimate Once: Unified, One-stage, Real-Time Category-level Articulated Object 6D Pose Estimation for Robotic Grasping](https://mlsanity.com/paper/2506.05719/) (2025)
- [DICArt: Advancing Category-level Articulated Object Pose Estimation in Discrete State-Spaces](https://mlsanity.com/paper/2602.19565/) (2026)
- [RaggeDi: Diffusion-based State Estimation of Disordered Rags, Sheets, Towels and Blankets](https://mlsanity.com/paper/2409.11831/) (2024)
- [Flat'n'Fold: A Diverse Multi-Modal Dataset for Garment Perception and Manipulation](https://mlsanity.com/paper/2409.18297/) (2024)
- [Learning a General Model: Folding Clothing with Topological Dynamics](https://mlsanity.com/paper/2504.20720/) (2025)
- [Beyond Static Perception: Integrating Temporal Context into VLMs for Cloth Folding](https://mlsanity.com/paper/2505.07600/) (2025)
- [ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusion](https://mlsanity.com/paper/2610.16293/) (2026)

## Abstract (arXiv)

Garments are important to humans. A visual system that can estimate and track the complete garment pose can be useful for many downstream tasks and real-world applications. In this work, we present a complete package to address the category-level garment pose tracking task: (1) A recording system VR-Garment, with which users can manipulate virtual garment models in simulation through a VR interface. (2) A large-scale dataset VR-Folding, with complex garment pose configurations in manipulation like flattening and folding. (3) An end-to-end online tracking framework GarmentTracking, which predicts complete garment pose both in canonical space and task space given a point cloud sequence. Extensive experiments demonstrate that the proposed GarmentTracking achieves great performance even when the garment has large non-rigid deformation. It outperforms the baseline approach on both speed and accuracy. We hope our proposed solution can serve as a platform for future research. Codes and datasets are available in https://garment-tracking.robotflow.ai.