# Worldscape-MoE: A Unified Mixture-of-Experts World Model for Scalable Heterogeneous Action Control

arXiv: https://arxiv.org/abs/2607.03964 · HTML: https://mlsanity.com/paper/2607.03964/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 44 of its 51 references that are in the map.
- First posted on arXiv 4 Jul 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)
- Mixture-of-experts / [MoE in reinforcement learning and robotics](https://mlsanity.com/moe/moe-in-reinforcement-learning-and-robotics/)

## Builds on (most cited first, 30 of 44)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning](https://mlsanity.com/paper/2306.03310/) (2023)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [Genie: Generative Interactive Environments](https://mlsanity.com/paper/2402.15391/) (2024)
- [MotionCtrl: A Unified and Flexible Motion Controller for Video Generation](https://mlsanity.com/paper/2312.03641/) (2023)
- [DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understanding](https://mlsanity.com/paper/2412.10302/) (2024)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [Reconstructing Hands in 3D with Transformers](https://mlsanity.com/paper/2312.05251/) (2023)
- [MoE-LLaVA: Mixture of Experts for Large Vision-Language Models](https://mlsanity.com/paper/2401.15947/) (2024)
- [EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video](https://mlsanity.com/paper/2505.11709/) (2025)
- [World Simulation with Video Foundation Models for Physical AI](https://mlsanity.com/paper/2511.00062/) (2025)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [Matrix-game 2.0: An open-source real-time and streaming interactive world model](https://mlsanity.com/paper/2508.13009/) (2025)
- [WorldPlay: Towards Long-Term Geometric Consistency for Real-Time Interactive World Modeling](https://mlsanity.com/paper/2512.14614/) (2025)
- [ViPE: Video Pose Engine for 3D Geometric Perception](https://mlsanity.com/paper/2508.10934/) (2025)
- [Ctrl-World: A Controllable Generative World Model for Robot Manipulation](https://mlsanity.com/paper/2510.10125/) (2025)
- [Understanding World or Predicting Future? A Comprehensive Survey of World Models](https://mlsanity.com/paper/2411.14499/) (2024)
- [HunyuanVideo 1.5 Technical Report](https://mlsanity.com/paper/2511.18870/) (2025)
- [Seedance 2.0: Advancing Video Generation for World Complexity](https://mlsanity.com/paper/2604.14148/) (2026)
- [AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers](https://mlsanity.com/paper/2411.18673/) (2024)
- [RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twins](https://mlsanity.com/paper/2504.13059/) (2025)
- [CamI2V: Camera-Controlled Image-to-Video Diffusion Model](https://mlsanity.com/paper/2410.15957/) (2024)
- [MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion](https://mlsanity.com/paper/2311.12052/) (2023)
- [Cosmos-Transfer1: Conditional World Generation with Adaptive Multimodal Control](https://mlsanity.com/paper/2503.14492/) (2025)

## Built on by (most cited first, 5 of 5)

- [IMPACT: Attention Is the Interaction Map for Scalable Interaction-Aware World Model Training](https://mlsanity.com/paper/2609.00161/) (2026)
- [CAER: Causal Action Effect Reweighting for World Model Training](https://mlsanity.com/paper/2608.30897/) (2026)
- [KineWorld: Action-Induced Transport Fields for Embodied World Modeling](https://mlsanity.com/paper/2610.06349/) (2026)
- [UniW(A)M: A Unified Mixture-of-Experts Framework for World and World-Action Models](https://mlsanity.com/paper/2610.18697/) (2026)
- [OneLA: Learning Many Worlds from Latent Actions](https://mlsanity.com/paper/2610.20160/) (2026)

## Abstract (arXiv)

World models are rapidly becoming a core infrastructure for embodied intelligence and interactive agents: they provide controllable simulators in which agents can perceive, act, forecast, and acquire scalable experience. Yet current video generation world models are still organized around isolated control interfaces, such as camera trajectories, robot actions, or hand-joint signals. This fragmentation is increasingly a scaling bottleneck. The central challenge is not the absence of controllable generators, but the lack of a unified and extensible learning framework that can absorb heterogeneous action supervision while preserving a shared model of world dynamics. In this work, we introduce Worldscape-MoE, a Mixture-of-Experts world model built on Diffusion Transformers for scalable heterogeneous action control. Our key observation is that different controls specify different interfaces to the same underlying world: although their representations differ, they constrain shared physical regularities, scene dynamics, and interaction semantics. Worldscape-MoE operationalizes this observation through modality-aware control injection, shared and control-specific experts, and a progressive MoE tuning strategy that supports continual extension to new action modalities. Experiments across locomotion, robotic manipulation, and egocentric hand control show that heterogeneous supervision improves rather than interferes with individual control capabilities. Worldscape-MoE achieves strong results on WorldArena, improves locomotion and hand-control metrics, exhibits robust out-of-distribution generalization, and demonstrates scaling behavior as additional control data and experts are integrated.