mlsanityData to 7 Oct 2026

OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints

arXiv 2501.03841 ↗First posted 7 Jan 2025cs.ROcited by 58

Not on the map. Why?

Abstract (arXiv)

The development of general robotic systems capable of manipulating in unstructured environments is a significant challenge. While Vision-Language Models(VLM) excel in high-level commonsense reasoning, they lack the fine-grained 3D spatial understanding required for precise manipulation tasks. Fine-tuning VLM on robotic datasets to create Vision-Language-Action Models(VLA) is a potential solution, but it is hindered by high data collection costs and generalization issues. To address these challenges, we propose a novel object-centric representation that bridges the gap between VLM's high-level reasoning and the low-level precision required for manipulation. Our key insight is that an object's canonical space, defined by its functional affordances, provides a structured and semantically meaningful way to describe interaction primitives, such as points and directions. These primitives act as a bridge, translating VLM's commonsense reasoning into actionable 3D spatial constraints. In this context, we introduce a dual closed-loop, open-vocabulary robotic manipulation system: one loop for high-level planning through primitive resampling, interaction rendering and VLM checking, and another for low-level execution via 6D pose tracking. This design ensures robust, real-time control without requiring VLM fine-tuning. Extensive experiments demonstrate strong zero-shot generalization across diverse robotic manipulation tasks, highlighting the potential of this approach for automating large-scale simulation data generation.

Key facts. Cited by 58 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Jan 2025.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 30; 2026: 28.

58 papers.

Sort:

By area:

  1. 2025A Survey on Vision-Language-Action Models: An Action Tokenization Perspectivecited by 96
  2. 2025DexGraspVLA: A Vision-Language-Action Framework Towards General Dexterous Graspingcited by 87
  3. 2025Any6D: Model-free 6D Pose Estimation of Novel Objectscited by 47
  4. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 47
  5. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 41
  6. 20253DFlowAction: Learning Cross-Embodiment Manipulation from 3D Flow World Modelcited by 36
  7. 2025Exploring the Limits of Vision-Language-Action Manipulations in Cross-task Generalizationcited by 31
  8. 2025Parallels Between VLA Model Post-Training and Human Motor Learning: Progress, Challenges, and Trendscited by 17
  9. 2025RoboDexVLM: Visual Language Model-Enabled Task Planning and Motion Control for Dexterous Robot Manipulationcited by 15
  10. 2025HumanoidGen: Data Generation for Bimanual Dexterous Manipulation via LLM Reasoningcited by 12
  11. 2025MimicFunc: Imitating Tool Manipulation from a Single Human Video via Functional Correspondencecited by 11
  12. 2025ODYSSEY: Open-World Quadrupeds Exploration and Manipulation for Long-Horizon Taskscited by 9
38 more
  1. 2025SafeBimanual: Diffusion-based Trajectory Optimization for Safe Bimanual Manipulationcited by 9
  2. 2025Percept-WAM: Perception-Enhanced World-Awareness-Action Model for Robust End-to-End Autonomous Drivingcited by 9
  3. 2025End-to-End Dexterous Arm-Hand VLA Policies via Shared Autonomy: VR Teleoperation Augmented by Autonomous Hand VLA Policy for Efficient Data Collectioncited by 8
  4. 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
  5. 2026RC-NF: Robot-Conditioned Normalizing Flow for Real-Time Anomaly Detection in Robotic Manipulationcited by 7
  6. 2026GuidedVLA: Specifying Task-Relevant Factors via Plug-and-Play Action Attention Specializationcited by 7
  7. 2026Guava: Distilling Frontier VLMs into a Compact Agent through a Robotic Manipulation Harnesscited by 7
  8. 2026FlowHOI: Flow-based Semantics-Grounded Generation of Hand-Object Interactions for Dexterous Robot Manipulationcited by 6
  9. 2026RoboRouter: Training-Free Policy Routing for Robotic Manipulationcited by 6
  10. 2026ETA: A New Agentic Paradigm for Embodied Taskscited by 6
  11. 2025Physically Ground Commonsense Knowledge for Articulated Object Manipulation with Analytic Conceptscited by 5
  12. 2025PRISM-DP: Spatial Pose-based Observations for Diffusion-Policies via Segmentation, Mesh Generation, and Pose Trackingcited by 5
  13. 2026AffordVLA: Injecting Affordance Representations into Vision-Language-Action Models via Implicit Feature Alignmentcited by 5
  14. 2025Robotic Manipulation via Imitation Learning: Taxonomy, Evolution, Benchmark, and Challengescited by 3
  15. 2025Executable Analytic Concepts as the Missing Link Between VLM Insight and Precise Manipulationcited by 3
  16. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 3
  17. 2026AffordGen: Generating Diverse Demonstrations for Generalizable Object Manipulation with Afford Correspondencecited by 2
  18. 2026From Imagined Futures to Executable Actions: Mixture of Latent Actions for Robot Manipulationcited by 2
  19. 2026ProxyPose: 6-DoF Pose Tracking via Video-to-Video Translationcited by 2
  20. 2025Foundation Feature-Driven Online End-Effector Pose Estimation: A Marker-Free and Learning-Free Approachcited by 1
  21. 2025RoboAct-CLIP: Video-Driven Pre-training of Atomic Action Understanding for Roboticscited by 1
  22. 2025T-Rex: Task-Adaptive Spatial Representation Extraction for Robotic Manipulation with Vision-Language Modelscited by 1
  23. 2025From Code to Action: Hierarchical Learning of Diffusion-VLM Policiescited by 1
  24. 2025IGen: Scalable Data Generation for Robot Learning from Open-World Imagescited by 1
  25. 2025Embodied Tree of Thoughts: Deliberate Manipulation Planning with Embodied World Modelcited by 1
  26. 2026Boosting SAM for Cross-Domain Few-Shot Segmentation via Conditional Point Sparsificationcited by 1
  27. 2026UniManip: General-Purpose Zero-Shot Robotic Manipulation with Agentic Operational Graphcited by 1
  28. 2026RealVLG-R1: A Large-Scale Real-World Visual-Language Grounding Benchmark for Robotic Perception and Manipulationcited by 1
  29. 2026CUBic: Coordinated Unified Bimanual Perception and Control Frameworkcited by 1
  30. 2026A Closed-Loop Multi-Agent Framework for Robust Multi-Robot Manipulationcited by 1
  31. 2026ManiSkillFormer: Demonstration-Free Compositional Manipulation via Geometric Contracts and Agentic Skill Graphcited by 1
  32. 2026World Action Agent: Harnessing VLMs for Robot Manipulation via World Action Rehearsalcited by 1
  33. 2025Object-Focus Actor for Data-efficient Robot Generalization Dexterous Manipulationcited by 0
  34. 2025FrankenBot: Brain-Morphic Modular Orchestration for Robotic Manipulation with Vision-Language Modelscited by 0
  35. 2025ImaginationPolicy: Towards Generalizable, Precise and Reliable End-to-End Policy for Robotic Manipulationcited by 0
  36. 2025GLUE: Global-Local Unified Encoding for Imitation Learning via Key-Patch Trackingcited by 0
  37. 2026TRACER: Texture-Robust Affordance Chain-of-Thought for Deformable-Object Refinementcited by 0
  38. 2026LAMP: Lift Image-Editing as General 3D Priors for Open-world Manipulationcited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulation
  2. 2024Task-oriented Robotic Manipulation with Vision Language Models
  3. 2024MOKA: Open-World Robotic Manipulation through Mark-Based Visual Prompting
  4. 2025GraphCoT-VLA: A 3D Spatial-Aware Reasoning Vision-Language-Action Model for Robotic Manipulation with Ambiguous Instructions
  5. 2024UniAff: A Unified Representation of Affordances for Tool Usage and Articulation with Vision-Language Models
  6. 2026BridgeVLA++: A Data-Efficient, Generalizable, and Memory-Augmented Vision-Language-Action Framework for 3D Manipulation
  7. 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
  8. 2025Improving Generalization of Language-Conditioned Robot Manipulation