mlsanityData to 14 Oct 2026

Out of Sight, Out of Mind? Evaluating State Evolution in Video World Models

arXiv 2603.13215 ↗First posted 13 Mar 2026cs.CVcited by 13

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Evolutions in the world, such as water pouring or ice melting, happen regardless of being observed. Video world models generate "worlds" via 2D frame observations. Can these generated "worlds" evolve regardless of observation? To probe this question, we design a benchmark to evaluate whether video world models can decouple state evolution from observation. Our benchmark, STEVO-Bench, applies observation control to evolving processes via instructions of occluder insertion, turning off the light, or specifying camera "lookaway" trajectories. By evaluating video models with and without camera control for a diverse set of naturally-occurring evolutions, we expose their limitations in decoupling state evolution from observation. STEVO-Bench proposes an evaluation protocol to automatically detect and disentangle failure modes of video world models across key aspects of natural state evolution. Analysis of STEVO-Bench results provide new insight into potential data and architecture bias of present-day video world models. Project website: https://glab-caltech.github.io/STEVOBench/. Blog: https://ziqi-ma.github.io/blog/2026/outofsight/

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 31 of its 39 references that are in the map. First posted on arXiv 13 Mar 2026.

Builds on

31 of 39 references.

Closest ancestors · 10

Same direction, newest first

  1. 2026Advancing Open-source World Models28 Jan
  2. 2026VerseCrafter: Dynamic Realistic Video World Model with 4D Geometric Control8 Jan
  3. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  4. 2025Video World Models with Long-term Spatial Memory5 Jun
  5. 2025Long-Context State-Space Video World Models26 May
  6. 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness27 Mar
  7. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  8. 2025Toward Stable World Models: Measuring and Addressing World Instability in Generative Environments11 Mar
  9. 2025WorldModelBench: Judging Video Generation Models As World Models28 Feb
  10. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug

Other references · 21

  1. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  2. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
  3. 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
  4. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
  5. 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
  6. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
  7. 2024Navigation World Modelscited by 272
  8. 2024VideoPhy: Evaluating Physical Commonsense for Video Generationcited by 228
  9. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  10. 2025VideoPhy-2: A Challenging Action-Centric Physical Commonsense Evaluation in Video Generationcited by 149
  11. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 128
  12. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 109
9 more
  1. 2025Aether: Geometric-Aware Unified World Modelingcited by 97
  2. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulationcited by 82
  3. 2025WonderPlay: Dynamic 3D Scene Generation from a Single Image and Actionscited by 45
  4. 2025Spatia: Video Generation with Updatable Spatial Memorycited by 40
  5. 2025PAI-Bench: A Comprehensive Benchmark For Physical AIcited by 34
  6. 2025BulletTime: Decoupled Control of Time and Camera Pose for Video Generationcited by 14
  7. 2026Flow Equivariant World Models: Memory for Partially Observed Dynamic Environmentscited by 8
  8. 2025World Consistency Score: A Unified Metric for Video Generation Qualitycited by 6
  9. 2025SmallWorlds: Assessing Dynamics Understanding of World Models in Isolated Environmentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 13.

13 papers.

Sort:

By area:

  1. 2026Current World Models Lack a Persistent State Corecited by 12
  2. 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearancecited by 6
  3. 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
  4. 2026CRONOS: Benchmarking Counterfactual Physical Consistency in Video Modelscited by 4
  5. 2026Teaching Video Generators to Remember: Eliciting Dynamic Memory for Out-of-Sight State Evolutioncited by 4
  6. 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
  7. 2026Remember to be Curious: Episodic Context and Persistent Worlds for 3D Explorationcited by 0
  8. 2026Can Image Models Imagine Time? ImageTime: A Novel Benchmark for Probing Visual World Modeling Through Spatiotemporal Consistencycited by 0
  9. 2026Can 4D Foundation Models Remember?cited by 0
  10. 2026Code Plans, Diffusion Renders: Open-Ended Generative World Modelingcited by 0
  11. 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
  12. 2026World Observer: Joint Actor-Observer Generation for Persistent World Modelingcited by 0
1 more
  1. 2026Grid or Nodes? Probing Object Memory in World Models Under Partial Observabilitycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models
  2. 2026StateCraft: Augmenting Video World Models with Abstract-State Reasoning
  3. 2026Video-Arena: A Scalable Agentic Video Benchmark for Spatiotemporal Capability
  4. 2026Can Video World Models Track Unobserved World States?
  5. 2026World Models Need a Physical Registry: Learning Internal Predictive States for Physically Plausible World Models
  6. 2026Omni-WorldBench: Towards a Comprehensive Interaction-Centric Evaluation for World Models
  7. 2026PAWBench: How Far Are We from Probabilistically Aligned World Modeling?
  8. 2026World in World: Explore the World with World Models