# Grasp-Oriented Fine-grained Cloth Segmentation without Real Supervision

arXiv: https://arxiv.org/abs/2110.02903 · HTML: https://mlsanity.com/paper/2110.02903/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 6 of 6)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Unsupervised Domain Adaptation by Backpropagation](https://mlsanity.com/paper/1409.7495/) (2014)
- [Learning a visuomotor controller for real world robotic grasping using simulated depth images](https://mlsanity.com/paper/1706.04652/) (2017)
- [Cloth Region Segmentation for Robust Grasp Selection](https://mlsanity.com/paper/2008.05626/) (2020)
- [Recognition of Grasp Points for Clothes Manipulation under unconstrained Conditions](https://mlsanity.com/paper/1706.06694/) (2017)
- [Multi-layer Domain Adaptation for Deep Convolutional Networks](https://mlsanity.com/paper/1909.02620/) (2019)

## Built on by (most cited first, 4 of 4)

- [InstructionBench: An Instructional Video Understanding Benchmark](https://mlsanity.com/paper/2504.05040/) (2025)
- [Semantic State Estimation in Cloth Manipulation Tasks](https://mlsanity.com/paper/2203.11647/) (2022)
- [Center Direction Network for Grasping Point Localization on Cloths](https://mlsanity.com/paper/2408.14456/) (2024)
- [Clothes Grasping and Unfolding Based on RGB-D Semantic Segmentation](https://mlsanity.com/paper/2305.03259/) (2023)

## Abstract (arXiv)

Automatically detecting graspable regions from a single depth image is a key ingredient in cloth manipulation. The large variability of cloth deformations has motivated most of the current approaches to focus on identifying specific grasping points rather than semantic parts, as the appearance and depth variations of local regions are smaller and easier to model than the larger ones. However, tasks like cloth folding or assisted dressing require recognising larger segments, such as semantic edges that carry more information than points. The first goal of this paper is therefore to tackle the problem of fine-grained region detection in deformed clothes using only a depth image. As a proof of concept, we implement an approach for T-shirts, and define up to 6 semantic regions of varying extent, including edges on the neckline, sleeve cuffs, and hem, plus top and bottom grasping points. We introduce a U-net based network to segment and label these parts. The second contribution of our work is concerned with the level of supervision that we require to train the proposed network. While most approaches learn to detect grasping points by combining real and synthetic annotations, in this work we defy the limitations of the synthetic data, and propose a multilayered domain adaptation (DA) strategy that does not use real annotations at all. We thoroughly evaluate our approach on real depth images of a T-shirt annotated with fine-grained labels. We show that training our network solely with synthetic data and the proposed DA yields results competitive with models trained on real data.