mlsanityData to 14 Oct 2026

EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models

arXiv 2406.05756 ↗First posted 9 Jun 2024cs.AI cs.CL cs.CVcited by 107

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

Abstract (arXiv)

The recent rapid development of Large Vision-Language Models (LVLMs) has indicated their potential for embodied tasks.However, the critical skill of spatial understanding in embodied environments has not been thoroughly evaluated, leaving the gap between current LVLMs and qualified embodied intelligence unknown. Therefore, we construct EmbSpatial-Bench, a benchmark for evaluating embodied spatial understanding of LVLMs.The benchmark is automatically derived from embodied scenes and covers 6 spatial relationships from an egocentric perspective.Experiments expose the insufficient capacity of current LVLMs (even GPT-4V). We further present EmbSpatial-SFT, an instruction-tuning dataset designed to improve LVLMs' embodied spatial understanding.

Key facts. Cited by 107 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 29 of its 48 references that are in the map. First posted on arXiv 9 Jun 2024.

Builds on

29 of 48 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023What's "up" with vision-language models? Investigating their struggle with spatial reasoning30 Oct
  2. 2023Multimodal Large Language Model for Visual Navigation12 Oct
  3. 2023PaLM-E: An Embodied Multimodal Language Model6 Mar

Other references · 26

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  6. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,117
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  9. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  10. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,040
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  12. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 906
14 more
  1. 2017AI2-THOR: An Interactive 3D Environment for Visual AIcited by 869
  2. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  3. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  4. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 236
  5. 2021TEACh: Task-driven Embodied Agents that Chatcited by 197
  6. 2023ImageBind-LLM: Multi-modality Instruction Tuningcited by 142
  7. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 138
  8. 2021Visual Room Rearrangementcited by 102
  9. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 77
  10. 2023What Matters in Training a GPT4-Style Language Model with Multimodal Inputs?cited by 64
  11. 2021Curriculum Learning for Vision-and-Language Navigationcited by 18
  12. 2022MVPTR: Multi-Level Semantic Alignment for Vision-Language Pre-Training via Multi-Stage Learningcited by 13
  13. 2023ReForm-Eval: Evaluating Large Vision Language Models via Unified Re-Formulation of Task-Oriented Benchmarkscited by 12
  14. 2024DELAN: Dual-Level Alignment for Vision-and-Language Navigation by Cross-Modal Contrastive Learningcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 32; 2026: 72.

107 papers.

Sort:

By area:

  1. 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
  2. 2026Qwen3.5-Omni Technical Reportcited by 174
  3. 2024RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Roboticscited by 136
  4. 2024SAT: Dynamic Spatial Aptitude Training for Multimodal Language Modelscited by 114
  5. 2025OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Modelscited by 114
  6. 2025Baichuan-Omni-1.5 Technical Reportcited by 111
  7. 2025STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding?cited by 79
  8. 2025MM-Spatial: Exploring 3D Spatial Understanding in Multimodal LLMscited by 73
  9. 2025Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenescited by 63
  10. 2025Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Modelscited by 52
  11. 2025SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulationcited by 46
  12. 2025How Far are VLMs from Visual Spatial Intelligence? A Benchmark-Driven Perspectivecited by 43
38 more
  1. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Spacecited by 39
  2. 2025Embodied-R1: Reinforced Embodied Reasoning for General Robotic Manipulationcited by 38
  3. 2025From Seeing to Doing: Bridging Reasoning and Decision for Robotic Manipulationcited by 35
  4. 2025Holistic Evaluation of Multimodal LLMs on Spatial Intelligencecited by 29
  5. 2025Spatial Understanding from Videos: Structured Prompts Meet Simulation Datacited by 28
  6. 2025TIGeR: Tool-Integrated Geometric Reasoning in Vision-Language Models for Roboticscited by 27
  7. 2025Robix: A Unified Model for Robot Interaction, Reasoning and Planningcited by 23
  8. 2026TSRBench: A Comprehensive Multi-task Multi-modal Time Series Reasoning Benchmark for Generalist Modelscited by 18
  9. 2026HY-Embodied-0.5: Embodied Foundation Models for Real-World Agentscited by 18
  10. 2026ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoningcited by 15
  11. 2024VISCO: Benchmarking Fine-Grained Critique and Correction Towards Self-Improvement in Visual Reasoningcited by 14
  12. 2025NaviTrace: Evaluating Embodied Navigation of Vision-Language Modelscited by 13
  13. 2026Wow, wo, val! A Comprehensive Embodied World Model Evaluation Turing Testcited by 13
  14. 2025Point-It-Out: Benchmarking Embodied Reasoning for Vision Language Models in Multi-Stage Visual Groundingcited by 10
  15. 2025Pelican-VL 1.0: A Foundation Brain Model for Embodied Intelligencecited by 10
  16. 2025Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Modelscited by 10
  17. 2025Is your VLM Sky-Ready? A Comprehensive Spatial Intelligence Benchmark for UAV Navigationcited by 10
  18. 2026Mage-VL: An Efficient Codec-Native Streaming Multimodal Foundation Modelcited by 10
  19. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 9
  20. 2026Hy-Embodied-VLM-1.0: Efficient Physical-World Agentscited by 8
  21. 2025Seeing is Not Reasoning: MVPBench for Graph-based Evaluation of Multi-path Visual Physical CoTcited by 6
  22. 2025LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Taskscited by 6
  23. 2026Do MLLMs Really Understand Space? A Mathematical Reasoning Evaluationcited by 6
  24. 2026SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Modelscited by 6
  25. 2026GEM: Generative Supervision Helps Embodied Intelligencecited by 6
  26. 2026OVO-S-Bench: A Hierarchical Benchmark for Streaming Spatial Intelligence in Multimodal LLMscited by 6
  27. 2026Embodied-R1.5: Evolving Physical Intelligence via Embodied Foundation Modelscited by 6
  28. 2026Paying More Attention to Visual Tokens in Self-Evolving Large Multimodal Modelscited by 6
  29. 2026SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environmentscited by 5
  30. 2026MOSS-VL Technical Reportcited by 5
  31. 2025EmbodiedBrain: Expanding Performance Boundaries of Task Planning for Embodied Intelligencecited by 4
  32. 2025Video2Layout: Recall and Reconstruct Metric-Grounded Cognitive Map for Spatial Reasoningcited by 4
  33. 2026Action-Sketcher: From Reasoning to Action via Visual Sketches for Long-Horizon Robotic Manipulationcited by 4
  34. 2026A2Eval: Agentic and Automated Evaluation for Embodied Braincited by 4
  35. 2026Orthogonal Model Mergingcited by 4
  36. 2026World2VLM: Distilling World Model Imagination into VLMs for Dynamic Spatial Reasoningcited by 4
  37. 2026Embodied3DBench: Benchmarking Low-Level Embodied Spatial Intelligence of Vision Language Modelscited by 4
  38. 2025VCM: Vision Concept Modeling Based on Implicit Contrastive Learning with Vision-Language Instruction Fine-Tuningcited by 3

50 most cited of 107 All 107 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025BLM1: A Boundless Large Model for Cross-Space, Cross-Task, and Cross-Embodiment Learning
  2. 2025Embodied Spatial Intelligence: from Implicit Scene Modeling to Spatial Reasoning
  3. 2025REM: Evaluating LLM Embodied Spatial Reasoning through Multi-Frame Trajectories
  4. 2026ESPIRE: A Diagnostic Benchmark for Embodied Spatial Reasoning of Vision-Language Models
  5. 2025UrbanVideo-Bench: Benchmarking Vision-Language Models on Embodied Intelligence with Video Data in Urban Spaces
  6. 2025Multimodal Spatial Reasoning in the Large Model Era: A Survey and Benchmarks
  7. 2026EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
  8. 2025Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning