# Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networks

arXiv: https://arxiv.org/abs/2012.03385 · HTML: https://mlsanity.com/paper/2012.03385/

## Key facts

- Cited by 77 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 6 Dec 2020.

## Built on by (most cited first, 30 of 77)

- [CLIPort: What and Where Pathways for Robotic Manipulation](https://mlsanity.com/paper/2109.12098/) (2021)
- [Habitat 2.0: Training Home Assistants to Rearrange their Habitat](https://mlsanity.com/paper/2106.14405/) (2021)
- [GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation](https://mlsanity.com/paper/2410.06158/) (2024)
- [3D Diffuser Actor: Policy Diffusion with 3D Scene Representations](https://mlsanity.com/paper/2402.10885/) (2024)
- [VIOLA: Imitation Learning for Vision-Based Manipulation with Object Proposal Priors](https://mlsanity.com/paper/2210.11339/) (2022)
- [Act3D: 3D Feature Field Transformers for Multi-Task Robotic Manipulation](https://mlsanity.com/paper/2306.17817/) (2023)
- [RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networks](https://mlsanity.com/paper/2205.02909/) (2022)
- [FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy](https://mlsanity.com/paper/2111.05623/) (2021)
- [Equivariant Descriptor Fields: SE(3)-Equivariant Energy-Based Models for End-to-End Visual Robotic Manipulation Learning](https://mlsanity.com/paper/2206.08321/) (2022)
- [SpeedFolding: Learning Efficient Bimanual Folding of Garments](https://mlsanity.com/paper/2208.10552/) (2022)
- [Diffusion-EDFs: Bi-equivariant Denoising Generative Modeling on SE(3) for Visual Robotic Manipulation](https://mlsanity.com/paper/2309.02685/) (2023)
- [Retrospectives on the Embodied AI Workshop](https://mlsanity.com/paper/2210.06849/) (2022)
- [IFOR: Iterative Flow Minimization for Robotic Object Rearrangement](https://mlsanity.com/paper/2202.00732/) (2022)
- [Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrations](https://mlsanity.com/paper/2408.04380/) (2024)
- [Fourier Transporter: Bi-Equivariant Robotic Manipulation in 3D](https://mlsanity.com/paper/2401.12046/) (2024)
- [Energy-based Models are Zero-Shot Planners for Compositional Scene Rearrangement](https://mlsanity.com/paper/2304.14391/) (2023)
- [Learning Foresightful Dense Visual Affordance for Deformable Object Manipulation](https://mlsanity.com/paper/2303.11057/) (2023)
- [ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation](https://mlsanity.com/paper/2505.09698/) (2025)
- [ClothesNet: An Information-Rich 3D Garment Model Repository with Simulated Clothes Environment](https://mlsanity.com/paper/2308.09987/) (2023)
- [UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondence](https://mlsanity.com/paper/2405.06903/) (2024)
- [VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulation](https://mlsanity.com/paper/2407.04152/) (2024)
- [ReorientDiff: Diffusion Model based Reorientation for Object Manipulation](https://mlsanity.com/paper/2303.12700/) (2023)
- [GarmentLab: A Unified Simulation and Benchmark for Garment Manipulation](https://mlsanity.com/paper/2411.01200/) (2024)
- [VisuoSpatial Foresight for Physical Sequential Fabric Manipulation](https://mlsanity.com/paper/2102.09754/) (2021)
- [Learning Deformable Object Manipulation from Expert Demonstrations](https://mlsanity.com/paper/2207.10148/) (2022)
- [Leveraging Symmetries in Pick and Place](https://mlsanity.com/paper/2308.07948/) (2023)
- [Learning Latent Graph Dynamics for Visual Manipulation of Deformable Objects](https://mlsanity.com/paper/2104.12149/) (2021)
- [Robots Enact Malignant Stereotypes](https://mlsanity.com/paper/2207.11569/) (2022)
- [Benchmarking the Sim-to-Real Gap in Cloth Manipulation](https://mlsanity.com/paper/2310.09543/) (2023)
- [UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Folding](https://mlsanity.com/paper/2311.01267/) (2023)

## Abstract (arXiv)

Rearranging and manipulating deformable objects such as cables, fabrics, and bags is a long-standing challenge in robotic manipulation. The complex dynamics and high-dimensional configuration spaces of deformables, compared to rigid objects, make manipulation difficult not only for multi-step planning, but even for goal specification. Goals cannot be as easily specified as rigid object poses, and may involve complex relative spatial relations such as "place the item inside the bag". In this work, we develop a suite of simulated benchmarks with 1D, 2D, and 3D deformable structures, including tasks that involve image-based goal-conditioning and multi-step deformable manipulation. We propose embedding goal-conditioning into Transporter Networks, a recently proposed model architecture for learning robotic manipulation that rearranges deep features to infer displacements that can represent pick and place actions. In simulation and in physical experiments, we demonstrate that goal-conditioned Transporter Networks enable agents to manipulate deformable structures into flexibly specified configurations without test-time visual anchors for target locations. We also significantly extend prior results using Transporter Networks for manipulating deformable objects by testing on tasks with 2D and 3D deformables. Supplementary material is available at https://berkeleyautomation.github.io/bags/.