mlsanityData to 7 Oct 2026

Multi-Agent Egocentric World Model with Fine-Grained Embodied Interaction

arXiv 2610.12299 ↗First posted 8 Oct 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Egocentric world models predict first-person observations conditioned on an agent's actions, but most focus on a single agent. Real embodied settings often involve multiple agents that act and interact within a shared environment. Existing multi-agent world models rely on coarse actions like locomotion, camera control, or discrete commands, leaving fine-grained embodied interactions underexplored. We formulate multi-agent egocentric world modeling as synchronized ego-stream generation for multiple agents interacting through fine-grained actions in a shared world. This requires cross-view action consistency, shared-environment consistency, and consistent propagation of interaction-induced state updates. We propose Multi-agent Egocentric World Model (ME-World), which jointly denoises multiple ego streams in a shared token sequence, conditions each stream on all agents' target-view poses, and grounds generation with shared environment memory. We train and evaluate on real and synthetic multi-agent data and introduce shared-world consistency metrics for environment, update, and identity consistency. Experiments show ME-World improves shared-world consistency, action control, identity preservation, and video quality over existing methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 41 of its 69 references that are in the map. First posted on arXiv 8 Oct 2026.

Builds on

41 of 69 references.

References · 41

  1. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  2. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2025DINOv3cited by 1,297
  5. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  6. 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
  7. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  8. 2023Ego-Exo4D: Understanding Skilled Human Activity from First- and Third-Person Perspectivescited by 477
  9. 2026World Action Models are Zero-shot Policiescited by 411
  10. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 337
  11. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  12. 2023Habitat 3.0: A Co-Habitat for Humans, Avatars and Robotscited by 220
29 more
  1. 2025World Simulation with Video Foundation Models for Physical AIcited by 183
  2. 2026Advancing Open-source World Modelscited by 144
  3. 2018Charades-Ego: A Large-Scale Dataset of Paired Third and First Person Videoscited by 127
  4. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 86
  5. 2025Hunyuan-GameCraft-2: Instruction-following Interactive Game World Modelcited by 58
  6. 2026Solaris: Building a Multiplayer Video World Model in Minecraftcited by 38
  7. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 33
  8. 2025World Models for Learning Dexterous Hand-Object Interactions from Human Videoscited by 26
  9. 2026MultiWorld: Scalable Multi-Agent Multi-View Video World Modelscited by 19
  10. 2026Multiplayer Interactive World Models with Representation Autoencoderscited by 14
  11. 2026AnyView: Synthesizing Any Novel View in Dynamic Scenescited by 12
  12. 2026Generated Reality: Human-centric World Simulation using Interactive Video Generation with Hand and Camera Controlcited by 12
  13. 2026MultiGen: Level-Design for Editable Multiplayer Worlds in Diffusion Game Enginescited by 12
  14. 2026EgoSim: Egocentric World Simulator for Embodied Interaction Generationcited by 11
  15. 2026Gamma-World: Generative Multi-Agent World Modeling Beyond Two Playerscited by 10
  16. 2026ShareVerse: Multi-Agent Consistent Video Generation for Shared World Modelingcited by 9
  17. 2026EgoForge: Goal-Directed Egocentric World Simulatorcited by 9
  18. 2026MetaWorld: Scaling Multi-Agent Video World Model from Single-view Video Datacited by 8
  19. 2026Prisma-World: Camera-Controllable Multi-Agent Video World Modelcited by 8
  20. 2026MASS: Multiplayer World Models with Authoritative Shared Statecited by 7
  21. 2026Streaming Multi-Agent Autoregressive Diffusion Model with World State Registerscited by 5
  22. 2026Rhythm: Learning Interactive Whole-Body Control for Dual Humanoidscited by 3
  23. 2026Controllable Egocentric Video Generation via Occlusion-Aware Sparse 3D Hand Jointscited by 3
  24. 2026E3C: Video Generation with 3D Environmental Memory and Ego-Exo Human Pose Controlcited by 2
  25. 2026AnchorWorld: Embodied Egocentric World Simulation with View-based Evolution Customizationcited by 2
  26. 2026HandsOnWorld: Unconstrained Egocentric Video Generation with Camera-Disentangled Hand Controlcited by 2
  27. 2026EgoGenesis: Egocentric World-Action Modeling with Online Anchored Projective Memory and Action-3D RoPEcited by 2
  28. 2026Population-Scalable Multi-Agent World Modelingcited by 2
  29. 2026ConsistWorld: Evidence Routing for Consistent Multi-Agent World Modelscited by 1

Built on by

Not cited yet.