Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction
Not on the map. Why?
Abstract (arXiv)
Egocentric world models predict first-person observations conditioned on an agent's actions, but most focus on a single agent. Real embodied settings often involve multiple agents that act and interact within a shared environment. Existing multi-agent world models rely on coarse actions like locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. We formulate multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions in a shared world. This requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates. We propose Multi-agent Egocentric World Model (ME-World), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents' target-view poses, and grounds generation with shared environment memory. We train and evaluate on real and synthetic multi-agent data and introduce shared-world consistency metrics for environment, update, and identity consistency. Experiments show ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 69 references that are in the map. First posted on arXiv 8 Oct 2026.
Builds on
41 of 69 references.
References · 41
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2017Decoupled Weight Decay Regularizationcited by 1,722
- 2025DINOv3cited by 1,297
- 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
- 2026World Action Models are Zero-shot Policiescited by 411
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
29 more
- 2025World Simulation with Video Foundation Models for Physical AIcited by 183
- 2026Advancing Open-source World Modelscited by 144
- 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
- 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
- 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 58
- 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
- 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 33
- 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 26
- 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 19
- 2026Multiplayer Interactive World Models with Representation Autoencoderscited by 14
- 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
- 2026Generated Reality: Human-centric World Simulation using Interactive Video Generation with Hand and Camera Controlcited by 12
- 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 12
- 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 11
- 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 10
- 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 9
- 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
- 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
- 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 8
- 2026MASS: Multiplayer World Models with Authoritative Shared Statecited by 7
- 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registerscited by 5
- 2026Rhythm: Learning Interactive Whole-Body Control for Dual Humanoidscited by 3
- 2026Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Jointscited by 3
- 2026E3C: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Controlcited by 2
- 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customizationcited by 2
- 2026HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Controlcited by 2
- 2026EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPEcited by 2
- 2026Population-Scalable Multi-Agent World Modelingcited by 2
- 2026ConsistWorld: Evidence Routing for Consistent Multi-Agent World Modelscited by 1
Built on by
Not cited yet.