mlsanityData to 14 Oct 2026

Do World Models Make Better Robots? A Survey of Evaluation Benchmarks for Predictive Embodied Intelligence

arXiv 2609.29669 ↗First posted 30 Aug 2026cs.RO cs.AIcited by 0

On the map: World models → World models for navigation

Abstract (arXiv)

Robot learning now advances along two tracks that rarely meet. On one side, direct Vision-Language-Action (VLA) policies map observations to actions and are scored by closed-loop task success. On the other, predictive and generative world models forecast future observations and are scored by open-loop prediction or generation quality. A natural question sits between them: does world modelling earn a measurable, closed-loop advantage over a direct policy, and for which robotic capabilities? We argue that the field cannot yet answer this question, and that the reason is a gap in how it is measured, not in the models themselves. World-model benchmarks score prediction without ever executing it, while task-success suites host a single policy and never build a world-model versus VLA contrast. This survey maps the evaluation landscape around that gap. We catalogue 160 web-verified benchmarks spanning 2017 to 2026 and organise them by evaluation mode, robotic capability, and model family into four lanes: policy suites, embodied agents, world model evaluation, and prediction-to-action bridges. Across the corpus, 138 of 160 benchmarks are model-agnostic and only 11 (7%) build an explicit VLA-versus-world-model contrast; counterfactual capability is almost entirely unmeasured, and only four benchmarks turn prediction into executed action. We contribute an operational taxonomy, a coverage comparison against the eight closest surveys (ours is the only one to cross capability with model family), an evaluation loop that isolates the advantage of prediction, and an actionable protocol of four advantage-aware metrics anchored on named testbeds. The organising claim is not that world models help or do not help, but that answering the question requires benchmarks built to ask it.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in World models for navigation, one of 11 directions in World models. Builds on 161 of its 164 references that are in the map. First posted on arXiv 30 Aug 2026.

Builds on

161 of 164 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026How Should World Models Be Evaluated for Embodied Decision-Making? A Decision-Making-Centric Position13 Jun
  2. 2026World Model for Robot Learning: A Comprehensive Survey30 Apr
  3. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models9 Feb
  4. 2024LEGENT: Open Platform for Embodied Agents28 Apr
  5. 2023Alexa Arena: A User-Centric Interactive Platform for Embodied AI2 Mar
  6. 2019Habitat: A Platform for Embodied AI Research2 Apr

Other references · 155

  1. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  2. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,451
  3. 2017CARLA: An Open Urban Driving Simulatorcited by 2,449
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  5. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  6. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,376
  7. 2020ALFWorld: Aligning Text and Embodied Environments for Interactive Learningcited by 1,205
  8. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  9. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  10. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  11. 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
  12. 2019ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Taskscited by 783
38 more
  1. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  2. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 689
  3. 2019The StarCraft Multi-Agent Challengecited by 597
  4. 2021Habitat-Matterport 3D Dataset (HM3D): 1000 Large-scale 3D Environments for Embodied AIcited by 592
  5. 2021CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Taskscited by 584
  6. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  7. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  8. 2020robosuite: A Modular Simulation Framework and Benchmark for Robot Learningcited by 511
  9. 2021Habitat 2.0: Training Home Assistants to Rearrange their Habitatcited by 511
  10. 2024Evaluating Real-World Robot Manipulation Policies in Simulationcited by 495
  11. 2019CLEVRER: CoLlision Events for Video REpresentation and Reasoningcited by 484
  12. 2018Gibson Env: Real-World Perception for Embodied Agentscited by 481
  13. 2020Beyond the Nav-Graph: Vision-and-Language Navigation in Continuous Environmentscited by 441
  14. 2018VirtualHome: Simulating Household Activities via Programscited by 415
  15. 2020Room-Across-Room: Multilingual Vision-and-Language Navigation with Dense Spatiotemporal Groundingcited by 414
  16. 2021Brax -- A Differentiable Physics Engine for Large Scale Rigid Body Simulationcited by 407
  17. 2022VIMA: General Robot Manipulation with Multimodal Promptscited by 407
  18. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  19. 2019Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learningcited by 384
  20. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  21. 2022ProcTHOR: Large-Scale Embodied AI Using Procedural Generationcited by 380
  22. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
  23. 2019On the Utility of Learning about Humans for Human-AI Coordinationcited by 349
  24. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  25. 2020Transporter Networks: Rearranging the Visual World for Robotic Manipulationcited by 344
  26. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  27. 2019REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environmentscited by 333
  28. 2023ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skillscited by 327
  29. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
  30. 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learningcited by 290
  31. 2021A Survey of Embodied AI: From Simulators to Research Taskscited by 261
  32. 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
  33. 2020ObjectNav Revisited: On Evaluation of Embodied Agents Navigating to Objectscited by 244
  34. 2018Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environmentscited by 239
  35. 2023Building Cooperative Embodied Agents Modularly with Large Language Modelscited by 237
  36. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 228
  37. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  38. 2023TidyBot: Personalized Robot Assistance with Large Language Modelscited by 221

50 most cited of 155 All 155 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform
  2. 2026World Action Models: The Next Frontier in Embodied AI
  3. 2026GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation
  4. 2025A Comprehensive Survey on World Models for Embodied AI
  5. 2026World Models for Embodied Intelligence: From Plausible to Controllable to Actionable
  6. 2026IndustrialVLA-Bench: A Traceable Multi-Axis Evaluation of Open Robot Policy Models
  7. 2026Think Like a World Model, Act Like a VLA: Distilling World-Model Representations into Compact Robot Policies
  8. 2026From World Action Models to Embodied Brains: A Roadmap for Open-World Physical Intelligence