# GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoning arXiv: https://arxiv.org/abs/2603.04158 ยท HTML: https://mlsanity.com/paper/2603.04158/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Builds on 30 of its 37 references that are in the map. - First posted on arXiv 4 Mar 2026. ## Builds on (most cited first, 30 of 30) - [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017) - [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024) - [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025) - [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017) - [GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Data](https://mlsanity.com/paper/2505.03233/) (2025) - [FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding](https://mlsanity.com/paper/2105.03655/) (2021) - [Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks](https://mlsanity.com/paper/2012.03385/) (2020) - [Learning to Manipulate Deformable Objects without Demonstrations](https://mlsanity.com/paper/1910.13439/) (2019) - [FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy](https://mlsanity.com/paper/2111.05623/) (2021) - [AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation](https://mlsanity.com/paper/2407.07889/) (2024) - [SpeedFolding: Learning Efficient Bimanual Folding of Garments](https://mlsanity.com/paper/2208.10552/) (2022) - [Graspness Discovery in Clutters for Fast and Accurate Grasp Detection](https://mlsanity.com/paper/2406.11142/) (2024) - [IFOR: Iterative Flow Minimization for Robotic Object Rearrangement](https://mlsanity.com/paper/2202.00732/) (2022) - [A Joint Modeling of Vision-Language-Action for Target-oriented Grasping in Clutter](https://mlsanity.com/paper/2302.12610/) (2023) - [Cloth Funnels: Canonicalized-Alignment for Multi-Purpose Garment Manipulation](https://mlsanity.com/paper/2210.09347/) (2022) - [Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation](https://mlsanity.com/paper/2303.11057/) (2023) - [Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experience](https://mlsanity.com/paper/2010.03209/) (2020) - [ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environment](https://mlsanity.com/paper/2308.09987/) (2023) - [UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence](https://mlsanity.com/paper/2405.06903/) (2024) - [GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation](https://mlsanity.com/paper/2411.01200/) (2024) - [One Policy to Dress Them All: Learning to Dress People with Diverse Poses and Garments](https://mlsanity.com/paper/2306.12372/) (2023) - [Where to relocate?: Object rearrangement inside cluttered and confined environments for robotic manipulation](https://mlsanity.com/paper/2003.10863/) (2020) - [Visual Foresight Trees for Object Retrieval from Clutter with Nonprehensile Rearrangement](https://mlsanity.com/paper/2105.02857/) (2021) - [UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Folding](https://mlsanity.com/paper/2311.01267/) (2023) - [Learning to Grasp Clothing Structural Regions for Garment Manipulation Tasks](https://mlsanity.com/paper/2306.14553/) (2023) - [Force-Constrained Visual Policy: Safe Robot-Assisted Dressing via Multi-Modal Sensing](https://mlsanity.com/paper/2311.04390/) (2023) - [Free-form language-based robotic reasoning and grasping](https://mlsanity.com/paper/2503.13082/) (2025) - [Benchmarking Generalizable Bimanual Manipulation: RoboTwin Dual-Arm Collaboration Challenge at CVPR 2025 MEIS Workshop](https://mlsanity.com/paper/2506.23351/) (2025) - [MetaFold: Language-Guided Multi-Category Garment Folding Framework via Trajectory Generation and Foundation Model](https://mlsanity.com/paper/2503.08372/) (2025) - [RoboHanger: Learning Generalizable Robotic Hanger Insertion for Diverse Garments](https://mlsanity.com/paper/2412.01083/) (2024) ## Built on by (most cited first, 1 of 1) - [AffordanceVLA: A Vision-Language-Action Model Empowering Action Generation through Affordance-Aware Understanding](https://mlsanity.com/paper/2606.06155/) (2026) ## Abstract (arXiv) Garment manipulation has attracted increasing attention due to its critical role in home-assistant robotics. However, the majority of existing garment manipulation works assume an initial state consisting of only one garment, while piled garments are far more common in real-world settings. To bridge this gap, we propose a novel garment retrieval pipeline that can not only follow language instruction to execute safe and clean retrieval but also guarantee exactly one garment is retrieved per attempt, establishing a robust foundation for the execution of downstream tasks (e.g., folding, hanging, wearing). Our pipeline seamlessly integrates vision-language reasoning with visual affordance perception, fully leveraging the high-level reasoning and planning capabilities of VLMs alongside the generalization power of visual affordance for low-level actions. To enhance the VLM's comprehensive awareness of each garment's state within a garment pile, we employ visual segmentation model (SAM2) to execute object segmentation on the garment pile for aiding VLM-based reasoning with sufficient visual cues. A mask fine-tuning mechanism is further integrated to address scenarios where the initial segmentation results are suboptimal. In addition, a dual-arm cooperation framework is deployed to address cases involving large or long garments, as well as excessive garment sagging caused by incorrect grasping point determination, both of which are strenuous for a single arm to handle. The effectiveness of our pipeline are consistently demonstrated across diverse tasks and varying scenarios in both real-world and simulation environments. Project page: https://garmentpile2.github.io/.