# WorldSimProbe: Diagnosing Simulator Faithfulness in Action-Conditioned World Models for Embodied Manipulation arXiv: https://arxiv.org/abs/2608.09298 · HTML: https://mlsanity.com/paper/2608.09298/ ## Key facts - Cited by 2 papers in the map (data through 14 Oct 2026). - Placed in Embodied and robot world models, one of 11 directions in World models. - Builds on 28 of its 38 references that are in the map. - First posted on arXiv 10 Aug 2026. ## Placed in - World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/) ## Builds on (most cited first, 28 of 28) - [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025) - [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025) - [WorldSimBench: Towards Video Generation Models as World Simulators](https://mlsanity.com/paper/2410.18072/) (2024) - [Cosmos World Foundation Model Platform for Physical AI](https://mlsanity.com/paper/2501.03575/) (2025) - [Learning Interactive Real-World Simulators](https://mlsanity.com/paper/2310.06114/) (2023) - [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026) - [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025) - [ManiSkill3: GPU Parallelized Robotics Simulation and Rendering for Generalizable Embodied AI](https://mlsanity.com/paper/2410.00425/) (2024) - [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025) - [Cosmos 3: Omnimodal World Models for Physical AI](https://mlsanity.com/paper/2606.02800/) (2026) - [DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos](https://mlsanity.com/paper/2602.06949/) (2026) - [IRASim: A Fine-Grained World Model for Robot Manipulation](https://mlsanity.com/paper/2406.14540/) (2024) - [WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models](https://mlsanity.com/paper/2602.08971/) (2026) - [WorldEval: World Model as Real-World Robot Policies Evaluator](https://mlsanity.com/paper/2505.19017/) (2025) - [WorldGym: World Model as An Environment for Policy Evaluation](https://mlsanity.com/paper/2506.00613/) (2025) - [EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models](https://mlsanity.com/paper/2505.09694/) (2025) - [Interactive World Simulator for Robot Policy Training and Evaluation](https://mlsanity.com/paper/2603.08546/) (2026) - [RoboWM-Bench: A Benchmark for Evaluating World Models in Robotic Manipulation](https://mlsanity.com/paper/2604.19092/) (2026) - [dWorldEval: Scalable Robotic Policy Evaluation via Discrete Diffusion World Model](https://mlsanity.com/paper/2604.22152/) (2026) - [GigaWorld-1: A Roadmap to Build World Models for Robot Policy Evaluation](https://mlsanity.com/paper/2607.02642/) (2026) - [ACWM-Phys: Investigating Generalized Physical Interaction in Action-Conditioned Video World Models](https://mlsanity.com/paper/2605.08567/) (2026) - [WorldArena 2.0: Extending Embodied World Model Benchmarking on Modality, Functionality and Platform](https://mlsanity.com/paper/2605.17912/) (2026) - [MiraBench: Evaluating Action-Conditioned Reliability in Robotic World Models](https://mlsanity.com/paper/2605.29360/) (2026) - [What-If World: A Causal Benchmark for General World Models in Embodied Scenarios](https://mlsanity.com/paper/2605.27589/) (2026) - [Hallucination in World Models is Predictable and Preventable](https://mlsanity.com/paper/2606.27326/) (2026) - [OSCAR: Omni-Embodiment Action-Conditioned World Model for Robotics](https://mlsanity.com/paper/2606.04463/) (2026) - [SC3-Eval: Evaluating Robot Foundation Models via Self-Consistent Video Generation](https://mlsanity.com/paper/2606.18610/) (2026) - [RoboTrustBench: Benchmarking the Trustworthiness of Video World Models for Robotic Manipulation](https://mlsanity.com/paper/2606.01600/) (2026) ## Built on by (most cited first, 2 of 2) - [World Models for Embodied Intelligence: From Plausible to Controllable to Actionable](https://mlsanity.com/paper/2609.16697/) (2026) - [RolloutFaith: Auditing Persistent Internal Interventions in Visual World Model](https://mlsanity.com/paper/2609.36843/) (2026) ## Abstract (arXiv) Action-conditioned world models (ACWMs) promise to provide embodied AI with scalable predictive simulators for planning, policy evaluation, and data generation. Realizing this promise requires precise action-conditioned transitions rather than merely plausible outputs. Yet their applicability remains difficult to establish because prevailing evaluations emphasize visual quality, task outcomes, or coarse rollout-level responsiveness without directly testing simulator fidelity. To address this gap, we evaluate ACWMs through the observable capabilities expected of physical simulators. Accordingly, we formalize Observable Simulator Contract, a minimal contract that any action-conditioned physical simulator should satisfy: supplied actions must induce corresponding agent motion, and environment responses must be grounded in that realized motion. To operationalize this contract, we introduce WorldSimProbe, comprising five controlled suites spanning local control sensitivity, global trajectory variation, source-diverse actions, interaction grounding, and dynamics. Suite-specific evaluators assess simulator-relative calibration, dense action-to-motion correspondence, false-interaction grounding, and primitive-level dynamics. We evaluate six open-source ACWMs on more than 18,000 instances across RoboTwin, ManiSkill, and LIBERO. World-SimProbe reveals systematic action-realization degradation across control variation, structured failures in interaction grounding and dynamics, and benchmark signals consistent with human judgments and downstream outcomes. Together, this capability-based framework provides a transparent, and standardized paradigm for diagnosing ACWM simulator fidelity beyond coarse, task-directed evaluation.