# Beyond Simulation: Benchmarking World Models for Planning and Causality in Autonomous Driving

arXiv: https://arxiv.org/abs/2508.01922 · HTML: https://mlsanity.com/paper/2508.01922/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 6 of its 32 references that are in the map.
- First posted on arXiv 3 Aug 2025.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)
- End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/)

## Builds on (most cited first, 6 of 6)

- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Mastering Atari with Discrete World Models](https://mlsanity.com/paper/2010.02193/) (2020)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning](https://mlsanity.com/paper/2109.12674/) (2021)
- [TrafficBots: Towards World Models for Autonomous Driving Simulation and Motion Prediction](https://mlsanity.com/paper/2303.04116/) (2023)

## Built on by (most cited first, 3 of 3)

- [Enactor: From Traffic Simulators to Surrogate World Models](https://mlsanity.com/paper/2603.18266/) (2026)
- [Validate the Dream Before You Trust Its Verdict: Admissibility for World-Model Simulators](https://mlsanity.com/paper/2607.07196/) (2026)
- [A Generative Model for Closed-Loop Microsimulation of Signalized Intersections](https://mlsanity.com/paper/2606.23588/) (2026)

## Abstract (arXiv)

World models have become increasingly popular in acting as learned traffic simulators. Recent work has explored replacing traditional traffic simulators with world models for policy training. In this work, we explore the robustness of existing metrics to evaluate world models as traffic simulators to see if the same metrics are suitable for evaluating a world model as a pseudo-environment for policy training. Specifically, we analyze the metametric employed by the Waymo Open Sim-Agents Challenge (WOSAC) and compare world model predictions on standard scenarios where the agents are fully or partially controlled by the world model (partial replay). Furthermore, since we are interested in evaluating the ego action-conditioned world model, we extend the standard WOSAC evaluation domain to include agents that are causal to the ego vehicle. Our evaluations reveal a significant number of scenarios where top-ranking models perform well under no perturbation but fail when the ego agent is forced to replay the original trajectory. To address these cases, we propose new metrics to highlight the sensitivity of world models to uncontrollable objects and evaluate the performance of world models as pseudo-environments for policy training and analyze some state-of-the-art world models under these new metrics.