Static and Plugged: Make Embodied Evaluation Simple
Not on the map. Why?
Abstract (arXiv)
Embodied intelligence is advancing rapidly, driving the need for efficient evaluation. Current benchmarks typically rely on interactive simulated environments or real-world setups, which are costly, fragmented, and hard to scale. To address this, we introduce StaticEmbodiedBench, a plug-and-play benchmark that enables unified evaluation using static scene representations. Covering 42 diverse scenarios and 8 core dimensions, it supports scalable and comprehensive assessment through a simple interface. Furthermore, we evaluate 19 Vision-Language Models (VLMs) and 11 Vision-Language-Action models (VLAs), establishing the first unified static leaderboard for Embodied intelligence. Moreover, we release a subset of 200 samples from our benchmark to accelerate the development of embodied intelligence.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 22 of its 31 references that are in the map. First posted on arXiv 6 Aug 2025.
Builds on
22 of 31 references.
References · 22
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2017Domain Randomization for Transferring Deep Neural Networks from Simulation to the Real Worldcited by 1,498
- 2019Habitat: A Platform for Embodied AI Researchcited by 1,191
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 665
- 2023BridgeData V2: A Dataset for Robot Learning at Scalecited by 657
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
10 more
- 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
- 2017Embodied Question Answeringcited by 336
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
- 2022The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrinkcited by 184
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
- 2024VLABench: A Large-Scale Benchmark for Language-Conditioned Robotics Manipulation with Long-Horizon Reasoning Taskscited by 175
- 2025EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agentscited by 57
- 2024LEGENT: Open Platform for Embodied Agentscited by 12
- 2025Image Quality Assessment for Embodied AIcited by 4
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2025EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents
- 2024EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models
- 2026How Foundational Skills Influence VLM-based Embodied Agents:A Native Perspective
- 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
- 2025ENACT: Evaluating Embodied Cognition with World Modeling of Egocentric Interaction
- 2026EPIC-Bench: A Perception-Centric Benchmark for Fine-Grained Embodied Visual Grounding in Vision-Language Models
- 2025Beyond Description: Cognitively Benchmarking Fine-Grained Action for Embodied Agents
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models