# Manipulating Highly Deformable Materials Using a Visual Feedback Dictionary

arXiv: https://arxiv.org/abs/1710.06947 · HTML: https://mlsanity.com/paper/1710.06947/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Oct 2017.

## Built on by (most cited first, 5 of 5)

- [Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation](https://mlsanity.com/paper/2303.11057/) (2023)
- [VisuoSpatial Foresight for Physical Sequential Fabric Manipulation](https://mlsanity.com/paper/2102.09754/) (2021)
- [Cloth Manipulation Using Random-Forest-Based Imitation Learning](https://mlsanity.com/paper/1802.09661/) (2018)
- [Co-manipulation of soft-materials estimating deformation from depth images](https://mlsanity.com/paper/2301.05609/) (2023)
- [Robotic Fabric Flattening with Wrinkle Direction Detection](https://mlsanity.com/paper/2303.04909/) (2023)

## Abstract (arXiv)

The complex physical properties of highly deformable materials such as clothes pose significant challenges fanipulation systems. We present a novel visual feedback dictionary-based method for manipulating defoor autonomous robotic mrmable objects towards a desired configuration. Our approach is based on visual servoing and we use an efficient technique to extract key features from the RGB sensor stream in the form of a histogram of deformable model features. These histogram features serve as high-level representations of the state of the deformable material. Next, we collect manipulation data and use a visual feedback dictionary that maps the velocity in the high-dimensional feature space to the velocity of the robotic end-effectors for manipulation. We have evaluated our approach on a set of complex manipulation tasks and human-robot manipulation tasks on different cloth pieces with varying material characteristics.