Data to 7 Oct 2026

Learning Predictive Representations for Deformable Objects Using Contrastive Estimation

arXiv 2003.05436 ↗First posted 11 Mar 2020cs.LG cs.CV cs.ROcited by 72

Not on the map. Why?

Abstract (arXiv)

Using visual model-based learning for deformable object manipulation is challenging due to difficulties in learning plannable visual representations along with complex dynamic models. In this work, we propose a new learning framework that jointly optimizes both the visual representation model and the dynamics model using contrastive estimation. Using simulation data collected by randomly perturbing deformable objects on a table, we learn latent dynamics models for these objects in an offline fashion. Then, using the learned models, we use simple model-based planning to solve challenging deformable object manipulation tasks such as spreading ropes and cloths. Experimentally, we show substantial improvements in performance over standard model-based learning techniques across our rope and cloth manipulation suite. Finally, we transfer our visual manipulation policies trained on data purely collected in simulation to a real PR2 robot through domain randomization.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 24; 2023: 14; 2024: 8; 2025: 6; 2026: 4.

72 papers.

Sort:

By area:

2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 359
2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 357
2021Reinforcement Learning with Prototypical Representationscited by 179
2022RoboCraft: Learning to See, Simulate, and Shape Elasto-Plastic Objects with Graph Networkscited by 69
2021Learning Visible Connectivity Dynamics for Cloth Smoothingcited by 61
2021Learning Task Informed Abstractionscited by 58
2023Self-Supervised Multimodal Learning: A Surveycited by 58
2021Robot Learning from Randomized Simulations: A Reviewcited by 56
2021FabricFlowNet: Bimanual Cloth Manipulation with a Flow-based Policycited by 54
2022Look Closer: Bridging Egocentric and Third-Person Views with Transformers for Robotic Manipulationcited by 49
2022Visual Reinforcement Learning with Self-Supervised 3D Representationscited by 45
2022DiffSkill: Skill Abstraction from Differentiable Physics for Deformable Object Manipulations with Toolscited by 44

38 more

2022Learning to Drive by Watching YouTube Videos: Action-Conditioned Contrastive Policy Pretrainingcited by 37
2023D3Fields: Dynamic 3D Descriptor Fields for Zero-Shot Generalizable Rearrangementcited by 35
2022Mesh-based Dynamics with Occlusion Reasoning for Cloth Manipulationcited by 32
2022DreamingV2: Reinforcement Learning with Discrete World Models without Reconstructioncited by 29
2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policycited by 25
2022ACID: Action-Conditional Implicit Visual Dynamics for Deformable Object Manipulationcited by 24
2023Learning World Models with Identifiable Factorizationcited by 23
2022Visuotactile Affordances for Cloth Manipulation with Local Controlcited by 20
2021VisuoSpatial Foresight for Physical Sequential Fabric Manipulationcited by 18
2021Learning Latent Graph Dynamics for Visual Manipulation of Deformable Objectscited by 17
2025Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulationcited by 16
2023Benchmarking the Sim-to-Real Gap in Cloth Manipulationcited by 14
2021Untangling Dense Non-Planar Knots by Learning Manipulation Features and Recovery Policiescited by 11
2022One After Another: Learning Incremental Skills for a Changing Worldcited by 10
2022EDO-Net: Learning Elastic Properties of Deformable Objects from Graph Dynamicscited by 10
2023Foldsformer: Learning Sequential Multi-Step Cloth Manipulation With Space-Time Attentioncited by 10
2021Learning Visual Shape Control of Novel 3D Deformable Objects from Partial-View Point Cloudscited by 9
2022Autonomously Untangling Long Cablescited by 9
2025Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Modelscited by 9
2023SculptBot: Pre-Trained Models for 3D Deformable Object Manipulationcited by 8
2022Weakly Supervised Correspondence Learningcited by 7
2022Learning Fabric Manipulation in the Real World with Human Videoscited by 7
2024SculptDiff: Learning Robotic Clay Sculpting from Humans with Goal Conditioned Diffusion Policycited by 7
2023Deep Reinforcement Learning Based on Local GNN for Goal-conditioned Deformable Object Rearrangingcited by 6
2022Multi-View Dreaming: Multi-View World Model with Contrastive Learningcited by 5
2022SGTM 2.0: Autonomously Untangling Long Cables using Interactive Perceptioncited by 5
2022PI-QT-Opt: Predictive Information Improves Multi-Task Robotic Reinforcement Learning at Scalecited by 5
2024Cross Domain Policy Transfer with Effect Cycle-Consistencycited by 5
2024General-purpose Clothes Manipulation with Semantic Keypointscited by 5
2025InstructionBench: An Instructional Video Understanding Benchmarkcited by 5
2021Enabling Visual Action Planning for Object Manipulation through Latent Space Roadmapcited by 4
2023DeformerNet: Learning Bimanual Manipulation of 3D Deformable Objectscited by 4
2023DexDLO: Learning Goal-Conditioned Dexterous Policy for Dynamic Manipulation of Deformable Linear Objectscited by 4
2021Unsupervised Feature Learning for Manipulation with Contrastive Domain Randomizationcited by 3
2021DeformerNet: A Deep Learning Approach to 3D Deformable Object Manipulationcited by 3
2021DiffSRL: Learning Dynamical State Representation for Deformable Object Manipulation with Differentiable Simulatorcited by 3
2022Tactile-Sensitive NewtonianVAE for High-Accuracy Industrial Connector Insertioncited by 3
2023GenDOM: Generalizable One-shot Deformable Object Manipulation with Parameter-Aware Policycited by 3

50 most cited of 72 All 72 (JSON)