mlsanityData to 7 Oct 2026

Grounding World Simulation Models in a Real-World Metropolis

arXiv 2603.15583 ↗First posted 16 Mar 2026cs.CVcited by 10

On the map: World models → Video-generation world models and simulators

Abstract (arXiv)

What if a world simulation model could render not an imagined environment but a city that actually exists? Prior generative world models synthesize visually plausible yet artificial environments by imagining all content. We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul. SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images. However, this design introduces several challenges, including temporal misalignment between retrieved references and the dynamic target scene, limited trajectory diversity and data sparsity from vehicle-mounted captures at sparse intervals. We address these challenges through cross-temporal pairing, a large-scale synthetic dataset enabling diverse camera trajectories, and a view interpolation pipeline that synthesizes coherent training videos from sparse street-view images. We further introduce a Virtual Lookahead Sink to stabilize long-horizon generation by continuously re-grounding each chunk to a retrieved image at a future location. We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor. SWM outperforms existing methods in generating spatially faithful, temporally consistent, long-horizon videos grounded in actual urban environments over trajectories reaching hundreds of meters, while supporting diverse camera movements and text-prompted scenario variations.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 59 of its 64 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

59 of 64 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026Advancing Open-source World Models28 Jan
  2. 2025Astra: General Interactive World Model with Autoregressive Denoising9 Dec
  3. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
  4. 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
  5. 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction25 Sep
  6. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
  7. 2025Video World Models with Long-term Spatial Memory5 Jun
  8. 2025DeepVerse: 4D Autoregressive Video Generation as a World Model1 Jun
  9. 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
  10. 2024Diffusion Models Are Real-Time Game Engines27 Aug
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  12. 2024UrbanWorld: An Urban World Model for 3D City Generation16 Jul
3 more
  1. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  2. 2022Scalable Diffusion Models with Transformers19 Dec
  3. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 44

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2025Qwen2.5-VL Technical Reportcited by 5,418
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  7. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  8. 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
  9. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  10. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  11. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  12. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
32 more
  1. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  2. 2018World Modelscited by 1,444
  3. 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
  4. 2025Qwen-Image Technical Reportcited by 973
  5. 2025SAM 3: Segment Anything with Conceptscited by 835
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  7. 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
  8. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  9. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  10. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
  11. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  12. 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
  13. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  14. 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
  15. 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
  16. 2025LongLive: Real-time Interactive Long Video Generationcited by 237
  17. 2024CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussianscited by 190
  18. 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
  19. 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
  20. 2025Aether: Geometric-Aware Unified World Modelingcited by 94
  21. 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
  22. 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 72
  23. 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
  24. 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
  25. 2024Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusioncited by 39
  26. 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 32
  27. 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
  28. 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
  29. 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
  30. 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
  31. 2026TeleStyle: Content-Preserving Style Transfer in Images and Videoscited by 5
  32. 2025Lookahead Anchoring: Preserving Character Identity in Audio-Driven Human Animationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 10.

10 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Sekai2: From World Exploration to Interactive World Modeling
  2. 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
  3. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms
  4. 2026World in World: Explore the World with World Models
  5. 2025Can World Models Benefit VLMs for World Dynamics?
  6. 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
  7. 2026Grounding Generated Videos in Feasible Plans via World Models
  8. 2025VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation