Grounding World Simulation Models in a Real-World Metropolis
On the map: World models → Video-generation world models and simulators
Abstract (arXiv)
What if a world simulation model could render not an imagined environment but a city that actually exists? Prior generative world models synthesize visually plausible yet artificial environments by imagining all content. We present Seoul World Model (SWM), a city-scale world model grounded in the real city of Seoul. SWM anchors autoregressive video generation through retrieval-augmented conditioning on nearby street-view images. However, this design introduces several challenges, including temporal misalignment between retrieved references and the dynamic target scene, limited trajectory diversity and data sparsity from vehicle-mounted captures at sparse intervals. We address these challenges through cross-temporal pairing, a large-scale synthetic dataset enabling diverse camera trajectories, and a view interpolation pipeline that synthesizes coherent training videos from sparse street-view images. We further introduce a Virtual Lookahead Sink to stabilize long-horizon generation by continuously re-grounding each chunk to a retrieved image at a future location. We evaluate SWM against recent video world models across three cities: Seoul, Busan, and Ann Arbor. SWM outperforms existing methods in generating spatially faithful, temporally consistent, long-horizon videos grounded in actual urban environments over trajectories reaching hundreds of meters, while supporting diverse camera movements and text-prompted scenario variations.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Video-generation world models and simulators, one of 11 directions in World models. Builds on 59 of its 64 references that are in the map. First posted on arXiv 16 Mar 2026.
Builds on
59 of 64 references.
Closest ancestors · 15
Same direction, newest first
- 2026Advancing Open-source World Models28 Jan
- 2025Astra: General Interactive World Model with Autoregressive Denoising9 Dec
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Model28 Nov
- 2025Rolling Forcing: Autoregressive Long Video Diffusion in Real Time29 Sep
- 2025FantasyWorld: Geometry-Consistent World Modeling via Unified Video and 3D Prediction25 Sep
- 2025Matrix-game 2.0: An open-source real-time and streaming interactive world model18 Aug
- 2025Video World Models with Long-term Spatial Memory5 Jun
- 2025DeepVerse: 4D Autoregressive Video Generation as a World Model1 Jun
- 2025Wan: Open and Advanced Large-Scale Video Generative Models26 Mar
- 2024Diffusion Models Are Real-Time Game Engines27 Aug
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024UrbanWorld: An Urban World Model for 3D City Generation16 Jul
3 more
Other references · 44
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
- 2017CARLA: An Open Urban Driving Simulatorcited by 2,431
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
- 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
32 more
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
- 2018World Modelscited by 1,444
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2025Qwen-Image Technical Reportcited by 973
- 2025SAM 3: Segment Anything with Conceptscited by 835
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2022Block-NeRF: Scalable Large Scene Neural View Synthesiscited by 720
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 618
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 392
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 325
- 2025LongLive: Real-time Interactive Long Video Generationcited by 237
- 2024CityGaussian: Real-time High-quality Large-Scale Scene Rendering with Gaussianscited by 190
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 157
- 2025VMem: Consistent Interactive Video Scene Generation with Surfel-Indexed View Memorycited by 102
- 2025Aether: Geometric-Aware Unified World Modelingcited by 94
- 2025Yume-1.5: A Text-Controlled Interactive World Generation Modelcited by 77
- 2024GenWarp: Single Image to Novel Views with Semantic-Preserving Generative Warpingcited by 72
- 2025Deep Forcing: Training-Free Long Video Generation with Deep Sink and Participative Compressioncited by 72
- 2025MotionStream: Real-Time Video Generation with Interactive Motion Controlscited by 71
- 2024Streetscapes: Large-scale Consistent Street View Generation Using Autoregressive Video Diffusioncited by 39
- 2025OmniNWM: Omniscient Driving Navigation World Modelscited by 32
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
- 2025WorldExplorer: Towards Generating Fully Navigable 3D Scenescited by 26
- 2024Multiagent Multitraversal Multimodal Self-Driving: Open MARS Datasetcited by 18
- 2026AnchorWeave: World-Consistent Video Generation with Retrieved Local Spatial Memoriescited by 13
- 2026TeleStyle: Content-Preserving Style Transfer in Images and Videoscited by 5
- 2025Lookahead Anchoring: Preserving Character Identity in Audio-Driven Human Animationcited by 4
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026WorldKV: Efficient World Memory with World Retrieval and Compressioncited by 14
- 2026TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignmentcited by 7
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trendscited by 6
- 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstructioncited by 2
- 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customizationcited by 2
- 2026FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matchingcited by 1
- 2026MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generationcited by 1
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?cited by 0
- 2026Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Modelscited by 0
- 2026World Observer: Joint Actor-Observer Generation for Persistent World Modelingcited by 0
- 2026World Observer: Joint Actor-Observer Generation for Persistent World Modeling1 Oct 2026
- 2026Learning What to Recall: Adaptive Multi-Cue Episodic Memory for World Models28 Sep 2026
- 2026MeRoPE: Metric Rotary Position Embedding for Camera-Controlled Video Generation1 Sep 2026
- 2026From Generation to Simulation: How Far Are World Models from Being True Simulators?24 Aug 2026
- 2026TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment11 Jun 2026
- 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customization5 Jun 2026
- 2026Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends31 May 2026
- 2026Learning Global Motion with Compact Gaussians for Feed-Forward 4D Reconstruction29 May 2026
- 2026WorldKV: Efficient World Memory with World Retrieval and Compression21 May 2026
- 2026FlowLong: Inference-time Long Video Generation via Manifold-constrained Tweedie Matching20 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026Sekai2: From World Exploration to Interactive World Modeling
- 2026INSPATIO-WORLD: A Real-Time 4D World Simulator via Spatiotemporal Autoregressive Modeling
- 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithms
- 2026World in World: Explore the World with World Models
- 2025Can World Models Benefit VLMs for World Dynamics?
- 2024WorldDreamer: Towards General World Models for Video Generation via Predicting Masked Tokens
- 2026Grounding Generated Videos in Feasible Plans via World Models
- 2025VDAWorld: World Modelling via VLM-Directed Abstraction and Simulation