DreamWAM: Beyond RGB Future Prediction for World Action Models
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30% to 98.40% and from 98.00% to 98.90%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36% to 63.44% and from 69.16% to 75.47%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4% across unseen changes in lighting, background, and object layout, compared with 55.6% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at https://github.com/hustvl/DreamWAM.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 40 of its 52 references that are in the map. First posted on arXiv 5 Aug 2026.
Builds on
40 of 52 references.
Closest ancestors · 11
Same direction, newest first
- 2026GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch15 Jul
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr
- 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems16 Apr
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model10 Feb
- 2026Causal World Modeling for Robot Control29 Jan
- 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation7 Jan
- 2025Motus: A Unified Latent Action World Model15 Dec
- 2025WorldVLA: Towards Autoregressive Action World Model26 Jun
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
Other references · 29
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
- 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
- 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
17 more
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
- 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
- 2025π*0.6: a VLA That Learns From Experiencecited by 280
- 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
- 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
- 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
- 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
- 2026A Pragmatic VLA Foundation Modelcited by 97
- 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
- 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoningcited by 44
- 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
- 2026BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generationcited by 25
- 2026DreamWorld: Unified World Modeling in Video Generationcited by 6
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026GlanceWAM: Sparse Test-Time Imagination for World-Action Modelscited by 1
- 2026MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generationcited by 0
- 2026CoRe-WAM: Correspondence-Aligned Temporal Residuals for World Action Modelscited by 0
- 2026FutureDuet: Decoupling Observation Access from Future Supervision in World Action Modelscited by 0
- 2026Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunkscited by 0
- 2026Rethinking Representations for World-Action Modelingcited by 0
- 2026Rethinking Representations for World-Action Modeling29 Sep 2026
- 2026Staircase Policy: Streaming Inference for World-Action Models with Large Action Chunks29 Sep 2026
- 2026FutureDuet: Decoupling Observation Access from Future Supervision in World Action Models28 Sep 2026
- 2026CoRe-WAM: Correspondence-Aligned Temporal Residuals for World Action Models23 Sep 2026
- 2026MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generation18 Sep 2026
- 2026GlanceWAM: Sparse Test-Time Imagination for World-Action Models25 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
- 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
- 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
- 2026From World Models to World Action Models: Rethinking Next-State Prediction
- 2026XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation
- 2026Learning Visual Feature-Based World Models via Residual Latent Action
- 2026Spatially Aware World Action Model via Geometric Latent Diffusion