# TRTM: Template-based Reconstruction and Target-oriented Manipulation of Crumpled Cloths

arXiv: https://arxiv.org/abs/2308.04670 · HTML: https://mlsanity.com/paper/2308.04670/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2023.

## Built on by (most cited first, 8 of 8)

- [Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulation](https://mlsanity.com/paper/2503.11999/) (2025)
- [Cloth-Splatting: 3D Cloth State Estimation from RGB Supervision](https://mlsanity.com/paper/2501.01715/) (2025)
- [TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approach](https://mlsanity.com/paper/2407.03245/) (2024)
- [InstructionBench: An Instructional Video Understanding Benchmark](https://mlsanity.com/paper/2504.05040/) (2025)
- [PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Robotics](https://mlsanity.com/paper/2410.07688/) (2024)
- [MobileManiBench: Simplifying Model Verification for Mobile Manipulation](https://mlsanity.com/paper/2602.05233/) (2026)
- [Beyond Static Perception: Integrating Temporal Context into VLMs for Cloth Folding](https://mlsanity.com/paper/2505.07600/) (2025)
- [ClothAtlas: Joint Garment Identification and State Estimation under Self-Occlusion](https://mlsanity.com/paper/2610.16293/) (2026)

## Abstract (arXiv)

Precise reconstruction and manipulation of the crumpled cloths is challenging due to the high dimensionality of cloth models, as well as the limited observation at self-occluded regions. We leverage the recent progress in the field of single-view human reconstruction to template-based reconstruct crumpled cloths from their top-view depth observations only, with our proposed sim-real registration protocols. In contrast to previous implicit cloth representations, our reconstruction mesh explicitly describes the positions and visibilities of the entire cloth mesh vertices, enabling more efficient dual-arm and single-arm target-oriented manipulations. Experiments demonstrate that our TRTM system can be applied to daily cloths that have similar topologies as our template mesh, but with different shapes, sizes, patterns, and physical properties. Videos, datasets, pre-trained models, and code can be downloaded from our project website: https://wenbwa.github.io/TRTM/ .