Thinker-WM: A Unified World Model with Shared Multimodal Representations for Embodied Intelligence
On the map: World models → Embodied and robot world models also fits: World models for navigation
Abstract (arXiv)
World models have emerged as a promising paradigm for embodied AI, yet existing systems often remain fragmented in the modalities they model and the functions they support. In particular, visual dynamics, geometric structure, and executable action are commonly coupled through separate interfaces or downstream modules rather than being jointly represented and evolved within a shared dynamics model. In this work, we present Thinker-WM, a unified multimodal world model that jointly models and generates RGB, metric depth, and robot action through a shared intermediate Unified Dynamics Transformer with modality-specific encoders and decoders. The model performs multimodal unification at the representation level: appearance, geometry, and action are transformed into a common dynamic representation and jointly evolved by shared parameters, while their distinct output structures are preserved. The same parameterization supports multiple functions, including joint video-action generation, world prediction, policy inference, action-conditioned generation, video-to-action generation, and inverse dynamics. To examine whether the shared representation captures action-related information, we introduce a lightweight motion probe that shows the frozen visual representation contains recoverable motion information. Thinker-WM is trained on a heterogeneous corpus comprising approximately 20K hours of egocentric video, 10K hours of real-robot data, and 1K hours of simulation data, including approximately 4K hours of real-robot data with aligned depth supervision. Extensive experiments demonstrate strong performance across embodied world-model evaluations. Thinker-WM ranks first on EWMBench, WorldOlympiad and WorldArena, while controlled studies further demonstrate the benefits of action conditioning, geometric interaction, and multi-function training within the shared representation.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 40 of its 44 references that are in the map. First posted on arXiv 13 Oct 2026.
Builds on
40 of 44 references.
Closest ancestors · 11
Same direction, newest first
- 2026Zero-WAM: In-Context World-Action Modeling from Human Videos for Open-Ended Task Generalization26 Aug
- 2026Xiaomi-Robotics-U0: Unified Embodied Synthesis with World Foundation Model13 Jul
- 2026RynnWorld-4D: 4D Embodied World Models for Robotic Manipulation7 Jul
- 2026Qwen-RobotWorld Technical Report: Unifying Embodied World Modeling through Language-Conditioned Video Generation15 Jun
- 2026WAM4D: Fast 4D World Action Model via Spatial Register Tokens12 Jun
- 2026GeoSem-WAM: Geometry- and Semantic-Aware World Action Models2 Jun
- 2026Motubrain: An Advanced World Action Model for Robot Control30 Apr
- 2026World Action Models are Zero-shot Policies17 Feb
- 2025Motus: A Unified Latent Action World Model15 Dec
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Models24 May
Other references · 29
- 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,556
- 2018World Modelscited by 1,513
- 2023Mastering Diverse Domains through World Modelscited by 1,253
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2024Genie: Generative Interactive Environmentscited by 747
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2023Learning Interactive Real-World Simulatorscited by 458
17 more
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 405
- 2024OpenVid-1M: A Large-Scale High-Quality Dataset for Text-to-video Generationcited by 341
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2024EgoMimic: Scaling Imitation Learning via Egocentric Videocited by 175
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 175
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2025RoboTwin: Dual-Arm Robot Benchmark with Generative Digital Twinscited by 121
- 2026EgoScale: Scaling Dexterous Manipulation with Diverse Egocentric Human Datacited by 94
- 2025SpatialTrackerV2: 3D Point Tracking Made Easycited by 87
- 2024Pandora: Towards General World Model with Natural Language Actions and Video Statescited by 86
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
- 2026EgoVerse: An Egocentric Human Dataset for Robot Learning from Around the Worldcited by 57
- 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 50
- 2025Fast-FoundationStereo: Real-Time Zero-Shot Stereo Matchingcited by 28
- 2026WorldOlympiad: Can Your World Model Survive a Triathlon?cited by 6
- 2026Diffusion ReRoll: Revisable Denoising for Robotic Sequential Predictioncited by 5
- 2025SplatPose: Geometry-Aware 6-DoF Pose Estimation from Single RGB Image via 3D Gaussian Splattingcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026LeapBot-WA: World-Anchor Action Models via Predictive Latent Alignments
- 2026EWAM: Emergent Depth-Wise Specialization in a Unified Embodied Model -- From Semantic Understanding through Visual Foresight to Action
- 2025A Comprehensive Survey on World Models for Embodied AI
- 2026WorldBagel: Uncovering the Power of Unified Multimodal Models for Vision-Language-Action-World Modeling
- 2026ViPSim: Collaborating Visual and Parameter Spaces for Consistent Long-Horizon Embodied World Models
- 2026Dreaming when Necessary: Advancing World Action Models with Adaptive Multi-Modal Reasoning
- 2026World Action Models: The Next Frontier in Embodied AI
- 2026Riemann-1.0: An Embodied World Action Model for Physical AI