mlsanityData to 14 Oct 2026

Visuotactile Affordances for Cloth Manipulation with Local Control

arXiv 2212.05108 ↗First posted 9 Dec 2022cs.RO cs.LG cs.SYcited by 21

Not on the map. Why?

Abstract (arXiv)

Cloth in the real world is often crumpled, self-occluded, or folded in on itself such that key regions, such as corners, are not directly graspable, making manipulation difficult. We propose a system that leverages visual and tactile perception to unfold the cloth via grasping and sliding on edges. By doing so, the robot is able to grasp two adjacent corners, enabling subsequent manipulation tasks like folding or hanging. As components of this system, we develop tactile perception networks that classify whether an edge is grasped and estimate the pose of the edge. We use the edge classification network to supervise a visuotactile edge grasp affordance network that can grasp edges with a 90% success rate. Once an edge is grasped, we demonstrate that the robot can slide along the cloth to the adjacent corner using tactile pose estimation/control in real time. See http://nehasunil.com/visuotactile/visuotactile.html for videos.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 22 of its 36 references that are in the map. First posted on arXiv 9 Dec 2022.

Builds on

22 of 36 references.

References · 22

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  4. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 264
  5. 2019TossingBot: Learning to Throw Arbitrary Objects with Residual Physicscited by 137
  6. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  7. 2018Learning Synergies between Pushing and Grasping with Self-supervised Deep Reinforcement Learningcited by 128
  8. 2017Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matchingcited by 125
  9. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 81
  10. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 77
  11. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  12. 2020Learning Rope Manipulation Policies Using Dense Object Descriptors Trained on Synthetic Depth Datacited by 51
10 more
  1. 2019Cable Manipulation with a Tactile-Reactive Grippercited by 46
  2. 2021GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completioncited by 43
  3. 2020VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulationcited by 40
  4. 2021GelSight Wedge: Measuring High-Resolution 3D Contact Geometry with a Compact Robot Fingercited by 34
  5. 2020Cloth Region Segmentation for Robust Grasp Selectioncited by 16
  6. 2021Occlusion-robust Deformable Object Tracking without Physics Simulationcited by 15
  7. 2018Tactile Regrasp: Grasp Adjustments via Simulated Tactile Transformationscited by 13
  8. 2020Learning Dense Visual Correspondences in Simulation to Smooth and Fold Real Fabricscited by 10
  9. 2022Semi-supervised 3D Object Detection via Temporal Graph Neural Networkscited by 10
  10. 2022Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Researchcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 3; 2024: 7; 2025: 6; 2026: 5.

21 papers.

Sort:

By area:

  1. 2023General In-Hand Object Rotation with Vision and Touchcited by 99
  2. 2024Learning Visuotactile Skills with Two Multifingered Handscited by 76
  3. 20243D-ViTac: Learning Fine-Grained Manipulation with Visuo-Tactile Sensingcited by 72
  4. 2024Transferable Tactile Transformers for Representation Learning Across Diverse Sensors and Taskscited by 45
  5. 2024UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondencecited by 23
  6. 2023GelSight Svelte: A Human Finger-shaped Single-camera Tactile Robot Finger with Large Sensing Coverage and Proprioceptive Sensingcited by 11
  7. 2024Sim2Real Manipulation on Unknown Objects with Tactile-based Reinforcement Learningcited by 11
  8. 2024FeelAnyForce: Estimating Contact Force Feedback from Tactile Sensation for Vision-Based Tactile Sensorscited by 11
  9. 2025VT-Refine: Learning Bimanual Assembly with Visuo-Tactile Feedback via Simulation Fine-Tuningcited by 11
  10. 2023SculptBot: Pre-Trained Models for 3D Deformable Object Manipulationcited by 8
  11. 2025Universal Visuo-Tactile Video Understanding for Embodied Interactioncited by 8
  12. 2026FlexiTac: A Low-Cost, Open-Source, Scalable Tactile Sensing Solution for Robotic Systemscited by 6
9 more
  1. 2026UniTacVLA: Unified Tactile Understanding and Prediction in Vision Language Action Modelscited by 5
  2. 2024LeTac-MPC: Learning Model Predictive Control for Tactile-reactive Graspingcited by 4
  3. 2025Reactive In-Air Clothing Manipulation with Confidence-Aware Dense Correspondence and Visuotactile Affordancecited by 4
  4. 2025TouchFormer: A Robust Transformer-based Framework for Multimodal Material Perceptioncited by 4
  5. 2025VTAO-BiManip: Masked Visual-Tactile-Action Pre-training with Object Understanding for Bimanual Dexterous Manipulationcited by 3
  6. 2025Force-Modulated Visual Policy for Robot-Assisted Dressing with Arm Motionscited by 2
  7. 2026ViTac-Tracing: Visual-Tactile Imitation Learning of Deformable Object Tracingcited by 0
  8. 2026VQ-Touch: A Data-Efficient Tactile Generation Framework Across Sensors and Scenarioscited by 0
  9. 2026Vision-Based 6-DoF Grasp Pose Estimation for Robot Cloth Unfoldingcited by 0