mlsanityData to 14 Oct 2026

Static and Plugged: Make Embodied Evaluation Simple

arXiv 2508.06553 ↗First posted 6 Aug 2025cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Embodied intelligence is advancing rapidly, driving the need for efficient evaluation. Current benchmarks typically rely on interactive simulated environments or real-world setups, which are costly, fragmented, and hard to scale. To address this, we introduce StaticEmbodiedBench, a plug-and-play benchmark that enables unified evaluation using static scene representations. Covering 42 diverse scenarios and 8 core dimensions, it supports scalable and comprehensive assessment through a simple interface. Furthermore, we evaluate 19 Vision-Language Models (VLMs) and 11 Vision-Language-Action models (VLAs), establishing the first unified static leaderboard for Embodied intelligence. Moreover, we release a subset of 200 samples from our benchmark to accelerate the development of embodied intelligence.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 6 Aug 2025.

Builds on

22 of 31 references.

References · 22

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  5. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  6. 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
  7. 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
  8. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
  9. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  10. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
  11. 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
  12. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
10 more
  1. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  2. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  3. 2017Embodied Question Answeringcited by 336
  4. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  5. 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
  6. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  7. 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 175
  8. 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 57
  9. 2024LEGENT: Open Platform for Embodied Agentscited by 12
  10. 2025Image Quality Assessment for Embodied AIcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
  2. 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models
  3. 2026How Foundational Skills Influence VLM-based Embodied Agents:A Native Perspective
  4. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
  5. 2025ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
  6. 2026EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
  7. 2025Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
  8. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models