Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Models
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
Abstract (arXiv)
Are current Vision Language Models (VLMs) ready to comprehend and reason about complex embodied interactions in 3D environments? We introduce Embodied3DBench, a robot-centric benchmark targeting low-level spatial intelligence in embodied 3D environments. To systematically evaluate these foundational perceptual capabilities, the benchmark includes 6 task categories divided into two core groups: Spatial Structural Understanding (Grounding, Spatial Relation Prediction, and Multi-view Correspondence) and Interaction-Oriented Perception (Affordance Prediction, Grasp Point Prediction, and Trajectory Prediction). The benchmark spans 12 subcategories and contains over 21k high-quality question-answer pairs. We evaluate 13 state-of-the-art models, and the results show that while current models exhibit relatively strong high-level spatial reasoning, such as understanding object-to-object positional relations, they remain fragile in interaction-oriented perception, highlighting a significant lack of robust 3D-aware interaction priors. To actively bridge this capability gap revealed by our benchmark, we further synthesize a large-scale training dataset comprising 1.3M QA pairs. Notably, fine-tuning on this dataset yields significant improvements in low-level spatial intelligence. Ultimately, Embodied3DBench fills a critical gap by providing both a systematic evaluation framework and a scalable data solution, setting a clear target for the development of interaction-aware multimodal systems.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 41 of its 55 references that are in the map. First posted on arXiv 27 May 2026.
Builds on
41 of 55 references.
Closest ancestors · 13
Same direction, newest first
- 2025RoboCerebra: A Large-scale Benchmark for Long-horizon Robotic Manipulation Evaluation7 Jun
- 2025VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction26 May
- 2025MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse24 Mar
- 2025RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete28 Feb
- 2025Mem2Ego: Empowering Vision-Language Models with Global-to-Ego Memory for Long-Horizon Embodied Navigation20 Feb
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces18 Dec
- 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models10 Dec
- 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics25 Nov
- 2024A3VLM: Actionable Articulation-Aware Vision Language Model11 Jun
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models9 Jun
- 2024SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models3 Jun
- 2024Kestrel: 3D Multimodal LLM for Part-Aware Grounded Description29 May
Other references · 28
- 2024GPT-4o System Cardcited by 4,588
- 2017PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Spacecited by 4,401
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
- 2023DriveLM: Driving with Graph Visual Question Answeringcited by 536
- 2018PartNet: A Large-scale Benchmark for Fine-grained and Hierarchical Part-level 3D Object Understandingcited by 526
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 519
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
- 2025Seed1.5-VL Technical Reportcited by 373
- 2024ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulationcited by 315
- 20163D Bounding Box Estimation Using Deep Learning and Geometrycited by 278
16 more
- 2022GAPartNet: Cross-Category Domain-Generalizable Object Perception and Manipulation via Generalizable and Actionable Partscited by 139
- 2025JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigationcited by 89
- 2025OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraintscited by 59
- 2025Evo-0: Vision-Language-Action Model with Implicit Spatial Understandingcited by 58
- 2024GenSim2: Scaling Robot Data Generation with Multi-modal and Reasoning LLMscited by 52
- 2022Domain Randomization-Enhanced Depth Simulation and Restoration for Perceiving and Grasping Specular and Transparent Objectscited by 50
- 2024Omni6DPose: A Benchmark and Model for Universal 6D Object Pose Estimation and Trackingcited by 50
- 2025ControlVLA: Few-shot Object-centric Adaptation for Pre-trained Vision-Language-Action Modelscited by 43
- 2025Mechanistic interpretability for steering vision-language-action modelscited by 31
- 2025CorrectNav: Self-Correction Flywheel Empowers Vision-Language-Action Navigation Modelcited by 27
- 2023GraspGF: Learning Score-based Grasping Primitive for Human-assisting Dexterous Graspingcited by 23
- 2024GEAL: Generalizable 3D Affordance Learning with Cross-Modal Consistencycited by 18
- 2023LVDiffusor: Distilling Functional Rearrangement Priors from Large Models into Diffusorcited by 7
- 2025CheckManual: A New Challenge and Benchmark for Manual-based Appliance Manipulationcited by 7
- 2026HiPolicy: Hierarchical Multi-Frequency Action Chunking for Policy Learningcited by 5
- 2026CADGrasp: Learning Contact and Collision Aware General Dexterous Grasping in Cluttered Scenescited by 4
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligencecited by 0
- 2026EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Modelscited by 0
- 2026ChronoGraph: Functional 4D Scene Graphs with Vision-Language Models for Interaction Understanding and Grounded Planningcited by 0
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoningcited by 0
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoning10 Oct 2026
- 2026ChronoGraph: Functional 4D Scene Graphs with Vision-Language Models for Interaction Understanding and Grounded Planning30 Sep 2026
- 2026EgoPathBench: Evaluating Zero-Shot Egocentric Waypoint Decision-Making in Vision-Language Models15 Sep 2026
- 2026Advantage-Guided Gate: Reshaping Open-Ended Reasoning for Vision-Based Spatial Intelligence8 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2025Abstract 3D Perception for Spatial Intelligence in Vision-Language Models
- 2025Embodied Spatial Intelligence: from Implicit Scene Modeling to Spatial Reasoning
- 2026SpatialPoint: Spatial-aware Point Prediction for Embodied Localization
- 2026EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
- 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Grounding
- 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
- 2025Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models