# Dynamic Manipulation of Deformable Objects using Imitation Learning with Adaptation to Hardware Constraints

arXiv: https://arxiv.org/abs/2403.12685 · HTML: https://mlsanity.com/paper/2403.12685/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 19 Mar 2024.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 15 of 15)

- [FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfolding](https://mlsanity.com/paper/2105.03655/) (2021)
- [Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks](https://mlsanity.com/paper/2012.03385/) (2020)
- [Iterative Residual Policy: for Goal-Conditioned Dynamic Manipulation of Deformable Objects](https://mlsanity.com/paper/2203.00663/) (2022)
- [Robots of the Lost Arc: Self-Supervised Learning to Dynamically Manipulate Fixed-Endpoint Cables](https://mlsanity.com/paper/2011.04840/) (2020)
- [DextAIRity: Deformable Manipulation Can be a Breeze](https://mlsanity.com/paper/2203.01197/) (2022)
- [Benchmarking the Sim-to-Real Gap in Cloth Manipulation](https://mlsanity.com/paper/2310.09543/) (2023)
- [AutoBag: Learning to Open Plastic Bags and Insert Objects](https://mlsanity.com/paper/2210.17217/) (2022)
- [Bag All You Need: Learning a Generalizable Bagging Strategy for Heterogeneous Objects](https://mlsanity.com/paper/2210.09997/) (2022)
- [Graph-based Task-specific Prediction Models for Interactions between Deformable and Rigid Objects](https://mlsanity.com/paper/2103.02932/) (2021)
- [Learning Visual Feedback Control for Dynamic Cloth Folding](https://mlsanity.com/paper/2109.04771/) (2021)
- [Bagging by Learning to Singulate Layers Using Interactive Perception](https://mlsanity.com/paper/2303.16898/) (2023)
- [QDP: Learning to Sequentially Optimise Quasi-Static and Dynamic Manipulation Primitives for Robotic Cloth Manipulation](https://mlsanity.com/paper/2303.13320/) (2023)
- [Sequential Topological Representations for Predictive Models of Deformable Objects](https://mlsanity.com/paper/2011.11693/) (2020)
- [A Reversible Dynamic Movement Primitive formulation](https://mlsanity.com/paper/2010.07708/) (2020)
- [ShakingBot: Dynamic Manipulation for Bagging](https://mlsanity.com/paper/2304.04558/) (2023)

## Abstract (arXiv)

Imitation Learning (IL) is a promising paradigm for learning dynamic manipulation of deformable objects since it does not depend on difficult-to-create accurate simulations of such objects. However, the translation of motions demonstrated by a human to a robot is a challenge for IL, due to differences in the embodiments and the robot's physical limits. These limits are especially relevant in dynamic manipulation where high velocities and accelerations are typical. To address this problem, we propose a framework that first maps a dynamic demonstration into a motion that respects the robot's constraints using a constrained Dynamic Movement Primitive. Second, the resulting object state is further optimized by quasi-static refinement motions to optimize task performance metrics. This allows both efficiently altering the object state by dynamic motions and stable small-scale refinements. We evaluate the framework in the challenging task of bag opening, designing the system BILBO: Bimanual dynamic manipulation using Imitation Learning for Bag Opening. Our results show that BILBO can successfully open a wide range of crumpled bags, using a demonstration with a single bag. See supplementary material at https://sites.google.com/view/bilbo-bag.