mlsanityData to 7 Oct 2026

FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policy

arXiv 2111.05623 ↗First posted 10 Nov 2021cs.RO cs.CVcited by 54

Not on the map. Why?

Abstract (arXiv)

We address the problem of goal-directed cloth manipulation, a challenging task due to the deformability of cloth. Our insight is that optical flow, a technique normally used for motion estimation in video, can also provide an effective representation for corresponding cloth poses across observation and goal images. We introduce FabricFlowNet (FFN), a cloth manipulation policy that leverages flow as both an input and as an action representation to improve performance. FabricFlowNet also elegantly switches between bimanual and single-arm actions based on the desired goal. We show that FabricFlowNet significantly outperforms state-of-the-art model-free and model-based cloth manipulation policies that take image input. We also present real-world experiments on a bimanual system, demonstrating effective sim-to-real transfer. Finally, we show that our method generalizes when trained on a single square cloth to other cloth shapes, such as T-shirts and rectangular cloths. Video and other supplementary materials are available at: https://sites.google.com/view/fabricflownet.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Builds on 24 of its 47 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

24 of 47 references.

References · 24

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020RAFT: Recurrent All-Pairs Field Transforms for Optical Flowcited by 2,385
  3. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  4. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  5. 2020SoftGym: Benchmarking Deep Reinforcement Learning for Deformable Object Manipulationcited by 118
  6. 2018Zero-Shot Visual Imitationcited by 114
  7. 2017Combining Self-Supervised Learning and Imitation for Vision-Based Rope Manipulationcited by 90
  8. 2021FlingBot: The Unreasonable Effectiveness of Dynamic Manipulation for Cloth Unfoldingcited by 80
  9. 2020Learning to Rearrange Deformable Cables, Fabrics, and Bags with Goal-Conditioned Transporter Networkscited by 77
  10. 2019Learning to Manipulate Deformable Objects without Demonstrationscited by 76
  11. 2020Learning Predictive Representations for Deformable Objects Using Contrastive Estimationcited by 72
  12. 2020Deep Imitation Learning for Bimanual Robotic Manipulationcited by 49
12 more
  1. 2021Tactile-RL for Insertion: Generalization to Objects of Unknown Geometrycited by 45
  2. 2019Efficient Bimanual Manipulation Using Learned Task Schemascited by 44
  3. 2019Deep Imitation Learning of Sequential Fabric Smoothing From an Algorithmic Supervisorcited by 44
  4. 2021GarmentNets: Category-Level Pose Estimation for Garments via Canonical Space Shape Completioncited by 41
  5. 2020VisuoSpatial Foresight for Multi-Step, Multi-Task Fabric Manipulationcited by 40
  6. 2019Dynamic Cloth Manipulation with Deep Reinforcement Learningcited by 26
  7. 2020Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulationcited by 25
  8. 2020Learning Arbitrary-Goal Fabric Folding with One Hour of Real Robot Experiencecited by 24
  9. 2020FlowControl: Optical Flow Based Visual Servoingcited by 15
  10. 2019A Grasping-centered Analysis for Cloth Manipulationcited by 14
  11. 2019Motion Perception in Reinforcement Learning with Dynamic Objectscited by 13
  12. 2020Intrinsic Motivation for Encouraging Synergistic Behaviorcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 10; 2023: 11; 2024: 14; 2025: 11; 2026: 8.

54 papers.

Sort:

By area:

  1. 2023RoboGen: Towards Unleashing Infinite Data for Automated Robot Learning via Generative Simulationcited by 206
  2. 2022FlowBot3D: Learning 3D Articulation Flow to Manipulate Articulated Objectscited by 91
  3. 2024General Flow as Foundation Affordance for Scalable Robot Learningcited by 80
  4. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 74
  5. 2023Multi-Stage Cable Routing through Hierarchical Imitation Learningcited by 59
  6. 2022SpeedFolding: Learning Efficient Bimanual Folding of Garmentscited by 51
  7. 2022DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Toolscited by 44
  8. 2023EquivAct: SIM(3)-Equivariant Visuomotor Policies beyond Rigid Object Manipulationcited by 37
  9. 2024Deep Generative Models in Robotics: A Survey on Learning from Multimodal Demonstrationscited by 35
  10. 2022Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulationcited by 32
  11. 2023Learning Foresightful Dense Visual Affordance for Deformable Object Manipulationcited by 25
  12. 2023DexDeform: Dexterous Deformable Object Manipulation with Human Demonstrations and Differentiable Physicscited by 23
38 more
  1. 2024UniGarmentManip: A Unified Framework for Category-Level Garment Manipulation via Dense Visual Correspondencecited by 22
  2. 2024VoxAct-B: Voxel-Based Acting and Stabilizing Policy for Bimanual Manipulationcited by 21
  3. 2025SKIL: Semantic Keypoint Imitation Learning for Generalizable Data-efficient Manipulationcited by 21
  4. 2025Dream2Flow: Bridging Video Generation and Open-World Manipulation with 3D Object Flowcited by 21
  5. 2024Render and Diffuse: Aligning Image and Action Spaces for Diffusion-based Behaviour Cloningcited by 19
  6. 2022DualAfford: Learning Collaborative Visual Affordance for Dual-gripper Manipulationcited by 18
  7. 2024ScrewMimic: Bimanual Imitation from Human Videos with Screw Space Projectioncited by 16
  8. 2024Neural Eulerian Scene Flow Fieldscited by 14
  9. 2023UniFolding: Towards Sample-efficient, Scalable, and Generalizable Robotic Garment Foldingcited by 13
  10. 2025Normalizing Flows are Capable Models for Continuous Controlcited by 13
  11. 2022EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamicscited by 10
  12. 2023Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attentioncited by 10
  13. 2024NaturalVLM: Leveraging Fine-grained Natural Language for Affordance-Guided Visual Manipulationcited by 10
  14. 2022Autonomously Untangling Long Cablescited by 9
  15. 2025GarmentPile: Point-Level Visual Affordance Guided Retrieval and Adaptation for Cluttered Garments Manipulationcited by 9
  16. 2022Learning Fabric Manipulation in the Real World with Human Videoscited by 7
  17. 2024SculptDiff: Learning Robotic Clay Sculpting from Humans with Goal Conditioned Diffusion Policycited by 7
  18. 2022Learning to Fold Real Garments with One Arm: A Case Study in Cloud-Based Robotics Researchcited by 6
  19. 2023DiffVL: Scaling Up Soft Body Manipulation using Vision-Language Driven Differentiable Physicscited by 6
  20. 2022Visual Haptic Reasoning: Estimating Contact Forces by Observing Deformable Object Interactionscited by 5
  21. 2023QDP: Learning to Sequentially Optimise Quasi-Static and Dynamic Manipulation Primitives for Robotic Cloth Manipulationcited by 5
  22. 2024TieBot: Learning to Knot a Tie from Visual Demonstration through a Real-to-Sim-to-Real Approachcited by 5
  23. 2024General-purpose Clothes Manipulation with Semantic Keypointscited by 5
  24. 2026SkillVLA: Tackling Combinatorial Diversity in Dual-Arm Manipulation via Skill Reusecited by 5
  25. 2025BiFold: Bimanual Cloth Folding with Language Guidancecited by 4
  26. 2025D-CODA: Diffusion for Coordinated Dual-Arm Data Augmentationcited by 4
  27. 2025One-Shot Affordance Grounding of Deformable Objects in Egocentric Organizing Scenescited by 3
  28. 2023Push Past Green: Learning to Look Behind Plant Foliage by Moving Itcited by 2
  29. 2024Non-rigid Relative Placement through 3D Dense Diffusioncited by 2
  30. 2023PlaNet-ClothPick: Effective Fabric Flattening Based on Latent Dynamic Planningcited by 1
  31. 2025CLASP: General-Purpose Clothes Manipulation with Semantic Keypointscited by 1
  32. 2025ROPA: Synthetic Robot Pose Generation for RGB-D Bimanual Data Augmentationcited by 1
  33. 2026GarmentPile++: Affordance-Driven Cluttered Garments Retrieval with Vision-Language Reasoningcited by 1
  34. 2026Instant-Fold: In-Context Imitation Learning for Deformable Object Manipulationcited by 1
  35. 2024DRAPER: Towards a Robust Robot Deployment and Reliable Evaluation for Quasi-Static Pick-and-Place Cloth-Shaping Neural Controllerscited by 0
  36. 2024UniRiT: Towards Few-Shot Non-Rigid Point Cloud Registrationcited by 0
  37. 2025Beyond Static Perception: Integrating Temporal Context into VLMs for Cloth Foldingcited by 0
  38. 2025RoboPearls: Editable Video Simulation for Robot Manipulationcited by 0

50 most cited of 54 All 54 (JSON)