Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence
On the map: World models → World models for navigation
Abstract (arXiv)
Robot learning now advances along two tracks that rarely meet. On one side, direct Vision-Language-Action (VLA) policies map observations to actions and are scored by closed-loop task success. On the other, predictive and generative world models forecast future observations and are scored by open-loop prediction or generation quality. A natural question sits between them: does world modelling earn a measurable, closed-loop advantage over a direct policy, and for which robotic capabilities? We argue that the field cannot yet answer this question, and that the reason is a gap in how it is measured, not in the models themselves. World-model benchmarks score prediction without ever executing it, while task-success suites host a single policy and never build a world-model versus VLA contrast. This survey maps the evaluation landscape around that gap. We catalogue 160 web-verified benchmarks spanning 2017 to 2026 and organise them by evaluation mode, robotic capability, and model family into four lanes: policy suites, embodied agents, world model evaluation, and prediction-to-action bridges. Across the corpus, 138 of 160 benchmarks are model-agnostic and only 11 (7%) build an explicit VLA-versus-world-model contrast; counterfactual capability is almost entirely unmeasured, and only four benchmarks turn prediction into executed action. We contribute an operational taxonomy, a coverage comparison against the eight closest surveys (ours is the only one to cross capability with model family), an evaluation loop that isolates the advantage of prediction, and an actionable protocol of four advantage-aware metrics anchored on named testbeds. The organising claim is not that world models help or do not help, but that answering the question requires benchmarks built to ask it.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 161 of its 164 references that are in the map. First posted on arXiv 30 Aug 2026.
Builds on
161 of 164 references.
Closest ancestors · 6
Same direction, newest first
- 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position13 Jun
- 2026World Model for Robot Learning: A Comprehensive Survey30 Apr
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models9 Feb
- 2024LEGENT: Open Platform for Embodied Agents28 Apr
- 2023Alexa Arena: A User-Centric Interactive Platform for Embodied AI2 Mar
- 2019Habitat: A Platform for Embodied AI Research2 Apr
Other references · 155
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
- 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
- 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
- 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
- 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
- 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
- 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
38 more
- 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
- 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
- 2019The StarCraft Multi-Agent Challengecited by 597
- 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
- 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
- 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
- 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
- 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
- 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
- 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
- 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
- 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
- 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
- 2018VirtualHome: Simulating Household Activities via Programscited by 415
- 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
- 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
- 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
- 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
- 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
- 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
- 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
- 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
- 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
- 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
- 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
- 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
- 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
- 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
- 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
- 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
- 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221
50 most cited of 155 All 155 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
- 2026World Action Models: The Next Frontier in Embodied AI
- 2026GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
- 2025A Comprehensive Survey on World Models for Embodied AI
- 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable
- 2026IndustrialVLA-Bench: A Traceable Multi-Axis Evaluation of Open Robot Policy Models
- 2026Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies
- 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence