MIND: Benchmarking Memory Consistency and Action Control in World Models
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
World models aim to understand, remember, and predict dynamic visual environments, yet a unified benchmark for evaluating their fundamental abilities remains lacking. To address this gap, we introduce MIND, the first open-domain closed-loop revisited benchmark for evaluating Memory consIstency and action coNtrol in worlD models. MIND contains 250 high-quality videos at 1080p and 24 FPS, including 100 (first-person) + 100 (third-person) video clips under a shared action space and 25 + 25 clips across varied action spaces covering eight diverse scenes. We design an efficient evaluation framework to measure two core abilities: memory consistency and action control, capturing temporal stability and contextual coherence across viewpoints. Furthermore, we design various action spaces, including different character movement speeds and camera rotation angles, to evaluate the action generalization capability across different action spaces under shared scenes. To facilitate future performance benchmarking on MIND, we introduce MIND-World, a novel interactive Video-to-World baseline. Extensive experiments demonstrate the completeness of MIND and reveal key challenges in current world models, including the difficulty of maintaining long-term memory consistency and generalizing across action spaces. Code: https://github.com/CSU-JPG/MIND.
Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 40 of its 48 references that are in the map. First posted on arXiv 8 Feb 2026.
Builds on
40 of 48 references.
Closest ancestors · 19
Same direction, newest first
- 2026Infinite-World: Scaling Interactive World Models to 1000-Frame Horizons via Pose-Free Hierarchical Memory2 Feb
- 2026Advancing Open-source World Models28 Jan
- 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
- 2025RELIC: Interactive Video World Model with Long-Horizon Memory3 Dec
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
- 2025Memory Forcing: Spatio-Temporal Memory for Consistent Scene Generation on Minecraft3 Oct
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Yume: An Interactive World Generation Model23 Jul
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025LoopNav: Benchmarking Spatial Consistency in World Models29 May
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulness27 Mar
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
7 more
- 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
- 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
- 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024Genie: Generative Interactive Environments23 Feb
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
Other references · 21
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
- 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 162
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
9 more
- 2025Long-Context Autoregressive Video Modeling with Next-Frame Predictioncited by 119
- 2025AdaWorld: Learning Adaptable World Models with Latent Actionscited by 108
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 85
- 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 66
- 2025Raw2Drive: Reinforcement Learning with Aligned World Models for End-to-End Autonomous Driving (in CARLA v2)cited by 45
- 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
- 2025Spatia: Video Generation with Updatable Spatial Memorycited by 33
- 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Modelscited by 17
Built on by
Citing papers per year: 2026: 31.
31 papers.
Sort:
By area:
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 24
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
- 2026Addressable Memory for Video World Modelscited by 11
- 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Modelscited by 9
- 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 8
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine Systemcited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
19 more
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
- 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearancecited by 5
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 4
- 2026From Pixels to States: Rethinking Interactive World Models as Game Enginescited by 4
- 2026World Models as Group Actionscited by 3
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worldscited by 3
- 2026ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadowcited by 2
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivitycited by 2
- 2026WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Modelscited by 2
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstructioncited by 1
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memorycited by 1
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?cited by 0
- 2026PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generationcited by 0
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Modelscited by 0
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Modelscited by 0
- 2026DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistencycited by 0
- 2026LOCI: Spatial Linear Memory for Streaming World Modelscited by 0
- 2026ROWBench: Do Video Models Render What the Program Specifies?cited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?cited by 0
- 2026MultiWorldBench: Do Independently Controlled Views Describe One Shared World?8 Oct 2026
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026LOCI: Spatial Linear Memory for Streaming World Models30 Sep 2026
- 2026DeCoPrune: Efficient KV-Cache Pruning for Autoregressive Video Diffusion via Denoising Consistency30 Sep 2026
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Models29 Sep 2026
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models28 Sep 2026
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory21 Sep 2026
- 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models2 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearance14 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
19 more
- 2026Addressable Memory for Video World Models7 Aug 2026
- 2026WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models5 Aug 2026
- 2026WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity3 Aug 2026
- 2026ShadowDancer: Teaching Video World Models Any Action by Learning Unified Dynamics Representations from a Video and Its Shadow30 Jul 2026
- 2026VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System29 Jul 2026
- 2026From Pixels to States: Rethinking Interactive World Models as Game Engines15 Jul 2026
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Models30 Jun 2026
- 2026PhysRAG: Enhancing Physics-Awareness in Video Generation via Retrieval-Augmented Generation25 Jun 2026
- 2026Current World Models Lack a Persistent State Core18 Jun 2026
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?9 Jun 2026
- 2026Dream.exe: Can Video Generation Models Dream Executable Robot Manipulation?3 Jun 2026
- 2026Geometry-Aware Implicit Memory for Video World Models1 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluation25 May 2026
- 2026World Models as Group Actions23 May 2026
- 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Models23 Apr 2026
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPG24 Mar 2026
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models7 Mar 2026
- 2026Olaf-World: Orienting Latent Actions for Video World Modeling10 Feb 2026
- 2026WBench: A Comprehensive Multi-turn Benchmark for Interactive Video World Model Evaluationcited by 34
- 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Modelscited by 24
- 2026LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Modelscited by 20
- 2026WildWorld: A Large-Scale Dataset for Dynamic World Modeling with Actions and Explicit State toward Generative ARPGcited by 13
- 2026Addressable Memory for Video World Modelscited by 11
- 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Modelscited by 9
- 2026Olaf-World: Orienting Latent Actions for Video World Modelingcited by 8
- 2026Current World Models Lack a Persistent State Corecited by 8
- 2026WorldRoamBench: An Open-World Benchmark for Long-Horizon Stability of Interactive World Modelscited by 8
- 2026Geometry-Aware Implicit Memory for Video World Modelscited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectivescited by 5
12 most cited of 26 All 26 (JSON)
- 2026ROWBench: Do Video Models Render What the Program Specifies?1 Oct 2026
- 2026LOCI: Spatial Linear Memory for Streaming World Models30 Sep 2026
- 2026Honeycomb: Constant-Size Scene Memory Representation for Video World Models29 Sep 2026
- 2026OPIS: An Input-Grounded Benchmark for Multi-Object Memory in Video World Models28 Sep 2026
- 2026WorldCrafter: Consistent Video World Model with Implicit 3D-aware Memory21 Sep 2026
- 2026SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models2 Sep 2026
- 2026RoboPhys-3D: A Comprehensive Embodied World Model Evaluation via 3D Reconstruction28 Aug 2026
- 2026HarnessEval-W: Agentifying the Evaluation of Visual Worlds17 Aug 2026
- 2026Marionette: Predicting World States, Rendering Geometry, Painting Appearance14 Aug 2026
- 2026PlayWorld: Benchmarking World Models with Agent Players over Long-Horizon Objectives13 Aug 2026
- 2026Addressable Memory for Video World Models7 Aug 2026
- 2026WorldCycle: Self-Verifiable Reinforcement Learning for Long-Horizon Video World Models5 Aug 2026
12 newest of 26 All 26 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026MBench: A Comprehensive Benchmark on Memory Capability for Video World Models
- 2026Echo-Memory: A Controlled Study of Memory in Action World Models
- 2026Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models
- 2026Wonder: Video World Model Done Better
- 2026ReWorld: An Interactive World Model with Long-Horizon Memory
- 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Models
- 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Framework
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Models