mlsanityData to 7 Oct 2026

SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models

arXiv 2605.23345 ↗First posted 22 May 2026cs.CVcited by 12

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

Interactive world models for first-person shooter (FPS) games must resolve high-frequency overlapping control signals at every frame without disrupting unaffected regions. Existing methods inject actions globally and train on single titles, failing under dense FPS inputs. We observe that FPS actions are spatially selective: discrete events such as firing or reloading affect only a localized region around the weapon (the scope), while continuous camera and movement signals govern stable surroundings. We propose SCOPE, which inserts a conditioning module into each transformer block of a pretrained video diffusion model. It reshapes features into per-pixel temporal sequences so that each position computes its action response from local visual content. This separates in-scope effects from out-of-scope generation without segmentation labels. We also introduce CrossFPS, the first multi-game FPS dataset with frame-aligned action telemetry. It comprises 69K clips from 7 titles with 10-DoF controller signals, curated to remove gameplay bias. The model learns general visual-to-action mappings rather than game-specific patterns, enabling zero-shot transfer to unseen scenes. Experiments confirm strong action responsiveness, precise scope separation, and effective cross-game generalization.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 51 of its 63 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

51 of 63 references.

Closest ancestors · 21

Same direction, newest first

  1. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer14 May
  2. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memory10 Apr
  3. 2026WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation17 Mar
  4. 2026Advancing Open-source World Models28 Jan
  5. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling16 Dec
  6. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  7. 2025MineWorld: a Real-Time and Open-Source Interactive World Model on Minecraft11 Apr
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  9. 2025GameFactory: Creating New Games with Generative Interactive Videos14 Jan
  10. 2024Open-Sora: Democratizing Efficient Video Production for All29 Dec
  11. 2024GameGen-X: Interactive Open-world Game Video Generation1 Nov
  12. 2024WorldSimBench: Towards Video Generation Models as World Simulators23 Oct
9 more
  1. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  2. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  3. 2024Genie: Generative Interactive Environments23 Feb
  4. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  5. 2023Learning Interactive Real-World Simulators9 Oct
  6. 2022Scalable Diffusion Models with Transformers19 Dec
  7. 2022Flow Matching for Generative Modeling6 Oct
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  9. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 30

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  5. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  6. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  7. 2018World Modelscited by 1,444
  8. 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,327
  9. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  10. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  11. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  12. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
18 more
  1. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Modelscited by 568
  2. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  3. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 332
  4. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 286
  5. 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 285
  6. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 161
  7. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  8. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  9. 2021DriveGAN: Towards a Controllable High-Quality Neural Simulationcited by 125
  10. 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Drivingcited by 89
  11. 2025WISA: World Simulator Assistant for Physics-Aware Text-to-Video Generationcited by 87
  12. 2020Learning to Simulate Dynamic Environments with GameGANcited by 86
  13. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 68
  14. 2025A Survey of Interactive Generative Videocited by 23
  15. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  16. 2026NitroGen: An Open Foundation Model for Generalist Gaming Agentscited by 22
  17. 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 21
  18. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026GameWAM: A World Action Model for Video Games
  2. 2026ActionParty: Multi-Subject Action Binding in Generative Video Games
  3. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model
  4. 2026EgoCS-400K: An Egocentric Gameplay Dataset for World Models
  5. 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Control
  6. 2026WorldCraft: From Camera Navigation to Object Manipulation in Interactive Video World Models
  7. 2026Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory
  8. 2026PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Models