mlsanityData to 7 Oct 2026

RoboCook: Long-Horizon Elasto-Plastic Object Manipulation with Diverse Tools

arXiv 2306.14447 ↗First posted 26 Jun 2023cs.ROcited by 75

Not on the map. Why?

Abstract (arXiv)

Humans excel in complex long-horizon soft body manipulation tasks via flexible tool use: bread baking requires a knife to slice the dough and a rolling pin to flatten it. Often regarded as a hallmark of human cognition, tool use in autonomous robots remains limited due to challenges in understanding tool-object interactions. Here we develop an intelligent robotic system, RoboCook, which perceives, models, and manipulates elasto-plastic objects with various tools. RoboCook uses point cloud scene representations, models tool-object interactions with Graph Neural Networks (GNNs), and combines tool classification with self-supervised policy learning to devise manipulation plans. We demonstrate that from just 20 minutes of real-world interaction data per tool, a general-purpose robot arm can learn complex long-horizon soft object manipulation tasks, such as making dumplings and alphabet letter cookies. Extensive evaluations show that RoboCook substantially outperforms state-of-the-art approaches, exhibits robustness against severe external disturbances, and demonstrates adaptability to different materials.

Key facts. Cited by 75 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Jun 2023.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 27; 2025: 25; 2026: 21.

75 papers.

Sort:

By area:

  1. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
  2. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  3. 2025DexVLA: Vision-Language Model with Plug-In Diffusion Expert for General Robot Controlcited by 191
  4. 2024SpatialBot: Precise Spatial Understanding with Vision Language Modelscited by 146
  5. 2024ShapeLLM: Universal 3D Object Understanding for Embodied Interactioncited by 133
  6. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 115
  7. 2025PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videoscited by 78
  8. 2024AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulationcited by 52
  9. 2024GIC: Gaussian-Informed Continuum for Physical Property Identification and Simulationcited by 48
  10. 2024Dynamic 3D Gaussian Tracking for Graph-Based Neural Dynamics Modelingcited by 42
  11. 2024PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulationcited by 36
  12. 2025BEHAVIOR Robot Suite: Streamlining Real-World Whole-Body Manipulation for Everyday Household Activitiescited by 29
38 more
  1. 2024GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policycited by 25
  2. 2025Particle-Grid Neural Dynamics for Learning Deformable Object Models from RGB-D Videoscited by 22
  3. 2025PUGS: Zero-shot Physical Understanding with Gaussian Splattingcited by 20
  4. 2024GarmentLab: A Unified Simulation and Benchmark for Garment Manipulationcited by 19
  5. 2024RoboPack: Learning Tactile-Informed Dynamics Models for Dense Packingcited by 18
  6. 2025Diffusion Dynamics Models with Generative State Estimation for Cloth Manipulationcited by 16
  7. 2025Learning Compositional Behaviors from Demonstration and Languagecited by 15
  8. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulationcited by 14
  9. 2024Point Cloud Models Improve Visual Robustness in Robotic Learnerscited by 13
  10. 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Modelscited by 13
  11. 2025ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulationcited by 13
  12. 2024Modeling the Real World with High-Density Visual Particle Dynamicscited by 12
  13. 2024Dynamics as Prompts: In-Context Learning for Sim-to-Real System Identificationscited by 10
  14. 2024DoughNet: A Visual Predictive Model for Topological Manipulation of Deformable Objectscited by 9
  15. 2025Learning Coordinated Bimanual Manipulation Policies using State Diffusion and Inverse Dynamics Modelscited by 9
  16. 2025VLMgineer: Vision Language Models as Robotic Toolsmithscited by 9
  17. 2025PhysWorld: From Real Videos to World Models of Deformable Objects via Physics-Aware Demonstration Synthesiscited by 9
  18. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibilitycited by 9
  19. 2023SculptBot: Pre-Trained Models for 3D Deformable Object Manipulationcited by 8
  20. 2024SculptDiff: Learning Robotic Clay Sculpting from Humans with Goal Conditioned Diffusion Policycited by 7
  21. 2024DeformPAM: Data-Efficient Learning for Long-horizon Deformable Object Manipulation via Preference-based Action Alignmentcited by 7
  22. 2025MASIV: Toward Material-Agnostic System Identification from Videoscited by 7
  23. 2024Differentiable Physics-based System Identification for Robotic Manipulation of Elastoplastic Materialscited by 6
  24. 2025Confounding Robust Deep Reinforcement Learning: A Causal Approachcited by 6
  25. 2025Improving Long-Range Interactions in Graph Neural Simulators via Hamiltonian Dynamicscited by 6
  26. 2026MOSIV: Multi-Object System Identification from Videoscited by 6
  27. 2024Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Featurescited by 4
  28. 2024PokeFlex: A Real-World Dataset of Volumetric Deformable Objects for Roboticscited by 4
  29. 2025RobotSmith: Generative Robotic Tool Design for Acquisition of Complex Manipulation Skillscited by 4
  30. 2025NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videoscited by 4
  31. 2025Open-Source Multimodal Moxin Models with Moxin-VLM and Moxin-VLAcited by 4
  32. 2024D-Cubed: Latent Diffusion Trajectory Optimisation for Dexterous Deformable Manipulationcited by 3
  33. 2024LIT: Large Language Model Driven Intention Tracking for Proactive Human-Robot Collaboration -- A Robot Sous-Chef Applicationcited by 3
  34. 2024Task Success Prediction for Open-Vocabulary Manipulation Based on Multi-Level Aligned Representationscited by 3
  35. 2025VisionLaw: Inferring Interpretable Intrinsic Dynamics from Visual Observations via Bilevel Optimizationcited by 3
  36. 2025Manipulating Elasto-Plastic Objects With 3D Occupancy and Learning-Based Predictive Controlcited by 2
  37. 2025Mash, Spread, Slice! Learning to Manipulate Object States via Visual Spatial Progresscited by 2
  38. 2026Confounding Robust Continuous Control via Automatic Reward Shapingcited by 2

50 most cited of 75 All 75 (JSON)