mlsanityData to 14 Oct 2026

Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

arXiv 2607.03964 ↗First posted 4 Jul 2026cs.RO cs.AIcited by 5

On the map: World models → Embodied and robot world models

also in 1 more
Abstract (arXiv)

World models are rapidly becoming a core infrastructure for embodied intelligence and interactive agents: they provide controllable simulators in which agents can perceive, act, forecast, and acquire scalable experience. Yet current video generation world models are still organized around isolated control interfaces, such as camera trajectories, robot actions, or hand-joint signals. This fragmentation is increasingly a scaling bottleneck. The central challenge is not the absence of controllable generators, but the lack of a unified and extensible learning framework that can absorb heterogeneous action supervision while preserving a shared model of world dynamics. In this work, we introduce Worldscape-MoE, a Mixture-of-Experts world model built on Diffusion Transformers for scalable heterogeneous action control. Our key observation is that different controls specify different interfaces to the same underlying world: although their representations differ, they constrain shared physical regularities, scene dynamics, and interaction semantics. Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy that supports continual extension to new action modalities. Experiments across locomotion, robotic manipulation, and egocentric hand control show that heterogeneous supervision improves rather than interferes with individual control capabilities. Worldscape-MoE achieves strong results on WorldArena, improves locomotion and hand-control metrics, exhibits robust out-of-distribution generalization, and demonstrates scaling behavior as additional control data and experts are integrated.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 44 of its 51 references that are in the map. First posted on arXiv 4 Jul 2026.

Builds on

44 of 51 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026LOME: Learning Human-Object Manipulation with Action-Conditioned Egocentric World Model28 Mar
  2. 2026Generated Reality: Human-centric World Simulation using Interactive Video Generation with Hand and Camera Control20 Feb
  3. 2026Hand2World: Autoregressive Egocentric Interaction Generation via Free-Space Hand Gestures10 Feb
  4. 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AI25 Nov
  5. 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulation11 Oct

Other references · 39

  1. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  2. 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  4. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,261
  5. 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
  6. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
  7. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  8. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  9. 2024Genie: Generative Interactive Environmentscited by 747
  10. 2023MotionCtrl: A Unified and Flexible Motion Controller for Video Generationcited by 580
  11. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  12. 2023DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Visioncited by 537
27 more
  1. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  2. 2023Reconstructing Hands in 3D with Transformerscited by 306
  3. 2024MoE-LLaVA: Mixture of Experts for Large Vision-Language Modelscited by 300
  4. 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
  5. 2025World Simulation with Video Foundation Models for Physical AIcited by 200
  6. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 198
  7. 2025Matrix-game 2.0: An open-source real-time and streaming interactive world modelcited by 186
  8. 2025ViPE: Video Pose Engine for 3D Geometric Perceptioncited by 170
  9. 2025WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modelingcited by 170
  10. 2024Understanding World or Predicting Future? A Comprehensive Survey of World Modelscited by 166
  11. 2025HunyuanVideo 1.5 Technical Reportcited by 160
  12. 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
  13. 2024AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformerscited by 154
  14. 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
  15. 2024CamI2V: Camera-Controlled Image-to-Video Diffusion Modelcited by 120
  16. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 117
  17. 2025Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Controlcited by 106
  18. 2026Matrix-Game 3.0: Real-Time and Streaming Interactive World Model with Long-Horizon Memorycited by 100
  19. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 80
  20. 2025SpatialVID: A Large-Scale Video Dataset with Spatial Annotationscited by 77
  21. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
  22. 2025Sekai: A Video Dataset towards World Explorationcited by 67
  23. 2025RealCam-I2V: Real-World Image-to-Video Generation with Interactive Complex Camera Controlcited by 57
  24. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 44
  25. 2025Astra: General Interactive World Model with Autoregressive Denoisingcited by 33
  26. 2026iWorld-Bench: A Benchmark for Interactive World Models with a Unified Action Generation Frameworkcited by 14
  27. 2026Nucleus-Image: Sparse MoE for Image Generationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026AHA-WAM:Asynchronous Horizon-Adaptive World-Action Modeling with Observation-Guided Context Routing
  2. 2026OpenWAM: An Open, Modular Exploration Towards Systematic World-Action Model Pretraining
  3. 2026MARS: Systematic Multi-Level Acceleration for World Action Models
  4. 2026World Action Models: The Next Frontier in Embodied AI
  5. 2026World Action Models are Zero-shot Policies
  6. 2026WorldMark: A Unified Benchmark Suite for Interactive Video World Models
  7. 2026World Tokens: Enhancing Embodied Policies with Training-Time World Modeling
  8. 2026ZimaBlue: Evolving Generalizable World Action Models through Scalable Video Pre-training