# MultiWorld: Scalable Multi-Agent Multi-View Video World Models

arXiv: https://arxiv.org/abs/2604.18564 · HTML: https://mlsanity.com/paper/2604.18564/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Placed in Video-generation world models and simulators, one of 11 directions in World models.
- Builds on 65 of its 72 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Placed in

- World models / [Video-generation world models and simulators](https://mlsanity.com/wm/video-generation-world-models-and-simulators/)

## Builds on (most cited first, 30 of 65)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Cameras](https://mlsanity.com/paper/2108.10869/) (2021)
- [Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusion](https://mlsanity.com/paper/2506.08009/) (2025)
- [CameraCtrl: Enabling Camera Control for Text-to-Video Generation](https://mlsanity.com/paper/2404.02101/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos](https://mlsanity.com/paper/2206.11795/) (2022)
- [Diffusion for World Modeling: Visual Details Matter in Atari](https://mlsanity.com/paper/2405.12399/) (2024)
- [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024)
- [WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion](https://mlsanity.com/paper/2312.07531/) (2023)
- [GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2503.20523/) (2025)
- [Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generation](https://mlsanity.com/paper/2602.02214/) (2026)
- [WorldMem: Long-term Consistent World Simulation with Memory](https://mlsanity.com/paper/2504.12369/) (2025)
- [Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrieval](https://mlsanity.com/paper/2506.03141/) (2025)
- [WorldScore: A Unified Evaluation Benchmark for World Generation](https://mlsanity.com/paper/2504.00983/) (2025)
- [Advancing Open-source World Models](https://mlsanity.com/paper/2601.20540/) (2026)
- [Video World Models with Long-term Spatial Memory](https://mlsanity.com/paper/2506.05284/) (2025)
- [GameFactory: Creating New Games with Generative Interactive Videos](https://mlsanity.com/paper/2501.08325/) (2025)

## Built on by (most cited first, 23 of 23)

- [Geometry-Aware Implicit Memory for Video World Models](https://mlsanity.com/paper/2606.02436/) (2026)
- [Prisma-World: Camera-Controllable Multi-Agent Video World Model](https://mlsanity.com/paper/2606.09507/) (2026)
- [MASS: Multiplayer World Models with Authoritative Shared State](https://mlsanity.com/paper/2608.06257/) (2026)
- [MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Data](https://mlsanity.com/paper/2606.02753/) (2026)
- [Kairos: A Regret-Aware Native World-Action Model Stack for Physical AI](https://mlsanity.com/paper/2606.16533/) (2026)
- [GEM-4D: Geometry-Enhanced Video World Models for Robot Manipulation](https://mlsanity.com/paper/2605.22882/) (2026)
- [Towards Interactive Video World Modeling: Frontiers, Challenges, Benchmarks, and Future Trends](https://mlsanity.com/paper/2606.01164/) (2026)
- [VideoCoCo: Code-as-CoT for Physically-Consistent Video Generation via an Agentic Dual-Engine System](https://mlsanity.com/paper/2607.27380/) (2026)
- [StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation](https://mlsanity.com/paper/2607.26754/) (2026)
- [Population-Scalable Multi-Agent World Modeling](https://mlsanity.com/paper/2608.08600/) (2026)
- [Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models](https://mlsanity.com/paper/2605.18431/) (2026)
- [WanToFight: Real-Time Generative Game Engine for Multi-Player Combat Interaction](https://mlsanity.com/paper/2607.12592/) (2026)
- [ConsistWorld: Evidence Routing for Consistent Multi-Agent World Models](https://mlsanity.com/paper/2609.22641/) (2026)
- [PhysEditWorld: A Large-Scale Dataset Toward Physics-Editable World Models](https://mlsanity.com/paper/2606.26694/) (2026)
- [GameDirector: Decoupling Gameplay Logic from Rendering for Player-Configurable Game World Models](https://mlsanity.com/paper/2609.25652/) (2026)
- [WorldCast: Distributed Multiplayer World Models](https://mlsanity.com/paper/2610.12412/) (2026)
- [Code Plans, Diffusion Renders: Open-Ended Generative World Modeling](https://mlsanity.com/paper/2609.26458/) (2026)
- [CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving](https://mlsanity.com/paper/2609.34749/) (2026)
- [Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update](https://mlsanity.com/paper/2610.07031/) (2026)
- [Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction](https://mlsanity.com/paper/2610.12299/) (2026)
- [Esports World Model Dataset](https://mlsanity.com/paper/2610.16166/) (2026)
- [Multics: Generative Multi-Player World Model with Hierarchical Communication](https://mlsanity.com/paper/2610.17201/) (2026)
- [PAWA: Panoramic Action-conditioned World Modeling for Multiple Agents](https://mlsanity.com/paper/2610.20240/) (2026)

## Abstract (arXiv)

Video world models have achieved remarkable success in simulating environmental dynamics in response to actions by users or agents. They are modeled as action-conditioned video generation models that take historical frames and current actions as input to predict future frames. Yet, most existing approaches are limited to single-agent scenarios and fail to capture the complex interactions inherent in real-world multi-agent systems. We present MultiWorld, a unified framework for multi-agent multi-view world modeling that enables accurate control of multiple agents while maintaining multi-view consistency. We introduce the Multi-Agent Condition Module to achieve precise multi-agent controllability, and the Global State Encoder to ensure coherent observations across different views. MultiWorld supports flexible scaling of agent and view counts, and synthesizes different views in parallel for high efficiency. Experiments on multi-player game environments and multi-robot manipulation tasks demonstrate that MultiWorld outperforms baselines in video fidelity, action-following ability, and multi-view consistency. Project page: https://multi-world.github.io/