mlsanityData to 7 Oct 2026

CoPa: General Robotic Manipulation through Spatial Constraints of Parts with Foundation Models

arXiv 2403.08248 ↗First posted 13 Mar 2024cs.ROcited by 97

Not on the map. Why?

Cited by 23 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Foundation models pre-trained on web-scale data are shown to encapsulate extensive world knowledge beneficial for robotic manipulation in the form of task planning. However, the actual physical implementation of these plans often relies on task-specific learning methods, which require significant data collection and struggle with generalizability. In this work, we introduce Robotic Manipulation through Spatial Constraints of Parts (CoPa), a novel framework that leverages the common sense knowledge embedded within foundation models to generate a sequence of 6-DoF end-effector poses for open-world robotic manipulation. Specifically, we decompose the manipulation process into two phases: task-oriented grasping and task-aware motion planning. In the task-oriented grasping phase, we employ foundation vision-language models (VLMs) to select the object's grasping part through a novel coarse-to-fine grounding mechanism. During the task-aware motion planning phase, VLMs are utilized again to identify the spatial geometry constraints of task-relevant object parts, which are then used to derive post-grasp poses. We also demonstrate how CoPa can be seamlessly integrated with existing robotic planning algorithms to accomplish complex, long-horizon tasks. Our comprehensive real-world experiments show that CoPa possesses a fine-grained physical understanding of scenes, capable of handling open-set instructions and objects with minimal prompt engineering and without additional training. Project page: https://copa-2024.github.io/

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Mar 2024.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 45; 2026: 28.

97 papers.

Sort:

By area:

  1. 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 305
  2. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agentscited by 213
  3. 2025DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledgecited by 203
  4. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 171
  5. 2025Magma: A Foundation Model for Multimodal AI Agentscited by 138
  6. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 133
  7. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 117
  8. 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
  9. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 105
  10. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  11. 2025InternVLA-M1: A Spatially Guided Vision-Language-Action Framework for Generalist Robot Policycited by 83
  12. 2024Manipulate-Anything: Automating Real-World Robots using Vision-Language Modelscited by 82
38 more
  1. 2024RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins (early version)cited by 68
  2. 2024Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detectioncited by 59
  3. 2024SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priorscited by 53
  4. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  5. 2024ImagineNav: Prompting Vision-Language Models as Embodied Navigator through Scene Imaginationcited by 38
  6. 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Modelcited by 36
  7. 2024Learning Precise Affordances from Egocentric Videos for Robotic Manipulationcited by 34
  8. 2024BUMBLE: Unifying Reasoning and Acting with Vision-Language Models for Building-wide Mobile Manipulationcited by 31
  9. 2025FP3: A 3D Foundation Policy for Robotic Manipulationcited by 31
  10. 2025ZeroMimic: Distilling Robotic Manipulation Skills from Web Videoscited by 31
  11. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  12. 2025RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluationcited by 31
  13. 2024CoA-VLA: Improving Vision-Language-Action Models via Visual-Textual Chain-of-Affordancecited by 29
  14. 2025UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulationcited by 23
  15. 2024Local Policies Enable Zero-shot Long-horizon Manipulationcited by 21
  16. 2025Robotic Visual Instructioncited by 21
  17. 2026PALM: Progress-Aware Policy Learning via Affordance Reasoning for Long-Horizon Robotic Manipulationcited by 20
  18. 2024Trust the PRoC3S: Solving Long-Horizon Robotics Problems with LLMs and Constraint Satisfactioncited by 19
  19. 2024PIVOT-R: Primitive-Driven Waypoint-Aware World Model for Robotic Manipulationcited by 17
  20. 2026AtomicVLA: Unlocking the Potential of Atomic Skill Learning in Robotscited by 15
  21. 2025Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbolscited by 14
  22. 2025Manual2Skill: Learning to Read Manuals and Acquire Robotic Skills for Furniture Assembly Using Vision-Language Modelscited by 13
  23. 2026RoboInter: A Holistic Intermediate Representation Suite Towards Robotic Manipulationcited by 12
  24. 2025RAGNet: Large-scale Reasoning-based Affordance Segmentation Benchmark towards General Graspingcited by 11
  25. 2024Open-World Task and Motion Planning via Vision-Language Model Generated Constraintscited by 9
  26. 2025SCA3D: Enhancing Cross-modal 3D Retrieval via 3D Shape and Caption Paired Data Augmentationcited by 9
  27. 2024Helpful DoggyBot: Open-World Object Fetching using Legged Robots and Vision-Language Modelscited by 8
  28. 2025CodeDiffuser: Attention-Enhanced Diffusion Policy via VLM-Generated Code for Instruction Ambiguitycited by 8
  29. 2025Maestro: Orchestrating Robotics Modules with Vision-Language Models for Zero-Shot Generalist Robotscited by 8
  30. 2026ST4VLA: Spatially Guided Training for Vision-Language-Action Modelscited by 8
  31. 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
  32. 2025SIMPACT: Simulation-Enabled Action Planning using Vision-Language Modelscited by 7
  33. 2026RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulationcited by 7
  34. 2024UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Modelscited by 6
  35. 2024One-Shot Manipulation Strategy Learning by Making Contact Analogiescited by 6
  36. 2025A Data-Centric Revisit of Pre-Trained Vision Models for Robot Learningcited by 6
  37. 2026PointACT: Vision-Language-Action Models with Multi-Scale Point-Action Interactioncited by 6
  38. 2026ETA: A New Agentic Paradigm for Embodied Taskscited by 6

50 most cited of 97 All 97 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints
  2. 2024Towards Open-World Grasping with Large Vision-Language Models
  3. 2025GeoManip: Geometric Constraints as General Interfaces for Robot Manipulation
  4. 2024FoundationGrasp: Generalizable Task-Oriented Grasping with Foundation Models
  5. 2026World Models for Robotic Manipulation: A Survey
  6. 2025PAC Bench: Do Foundation Models Understand Prerequisites for Executing Manipulation Policies?
  7. 2023Lan-grasp: Using Large Language Models for Semantic Object Grasping and Placement
  8. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation