# Model-Driven Feed-Forward Prediction for Manipulation of Deformable Objects

arXiv: https://arxiv.org/abs/1607.04411 · HTML: https://mlsanity.com/paper/1607.04411/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2016.

## Built on by (most cited first, 6 of 6)

- [Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulation](https://mlsanity.com/paper/2206.02881/) (2022)
- [ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulation](https://mlsanity.com/paper/2203.06856/) (2022)
- [Deformable Linear Object Prediction Using Locally Linear Latent Dynamics](https://mlsanity.com/paper/2103.14184/) (2021)
- [Semantic State Estimation in Cloth Manipulation Tasks](https://mlsanity.com/paper/2203.11647/) (2022)
- [Localization and Tracking of User-Defined Points on Deformable Objects for Robotic Manipulation](https://mlsanity.com/paper/2105.09067/) (2021)
- [Recognising Known Configurations of Garments For Dual-Arm Robotic Flattening](https://mlsanity.com/paper/2205.00225/) (2022)

## Abstract (arXiv)

Robotic manipulation of deformable objects is a difficult problem especially because of the complexity of the many different ways an object can deform. Searching such a high dimensional state space makes it difficult to recognize, track, and manipulate deformable objects. In this paper, we introduce a predictive, model-driven approach to address this challenge, using a pre-computed, simulated database of deformable object models. Mesh models of common deformable garments are simulated with the garments picked up in multiple different poses under gravity, and stored in a database for fast and efficient retrieval. To validate this approach, we developed a comprehensive pipeline for manipulating clothing as in a typical laundry task. First, the database is used for category and pose estimation for a garment in an arbitrary position. A fully featured 3D model of the garment is constructed in real-time and volumetric features are then used to obtain the most similar model in the database to predict the object category and pose. Second, the database can significantly benefit the manipulation of deformable objects via non-rigid registration, providing accurate correspondences between the reconstructed object model and the database models. Third, the accurate model simulation can also be used to optimize the trajectories for manipulation of deformable objects, such as the folding of garments. Extensive experimental results are shown for the tasks above using a variety of different clothing.