mlsanityData to 7 Oct 2026

DreamWAM: Beyond RGB Future Prediction for World Action Models

arXiv 2608.04996 ↗First posted 5 Aug 2026cs.ROcited by 6

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World Action Models (WAMs) learn action-relevant representations by predicting how the observed world will evolve. Most existing WAMs define this future in RGB space, where task-relevant state transitions are entangled with nuisance variations in texture, illumination, background, and viewpoint. We argue that WAMs should explicitly predict action-relevant future state rather than relying on RGB prediction alone. We introduce DreamWAM, which reformulates future prediction as structured world modeling beyond RGB, representing future states through complementary views of appearance, motion, geometry, and semantics. During training, DreamWAM combines joint latent denoising of RGB and motion with lightweight gated residual branches for geometry and semantics. Shared attention between VideoDiT and ActionDiT allows the action branch to learn from these future-state predictions, while all beyond-RGB supervision branches are disabled at inference and deployment remains RGB-only. Across both no-rollout and joint video-action inference, DreamWAM consistently improves the matched RGB-only baselines on LIBERO, from 97.30% to 98.40% and from 98.00% to 98.90%, respectively. The gains become larger under unseen LIBERO-Plus perturbations, from 51.36% to 63.44% and from 69.16% to 75.47%. The same robustness extends to real-world manipulation, where DreamWAM attains an average success rate of 74.4% across unseen changes in lighting, background, and object layout, compared with 55.6% for Fast-WAM-Joint. These results show that robust world-action learning depends not only on predicting the future, but on representing it in a form that matters for action. The code and models are publicly released at https://github.com/hustvl/DreamWAM.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 40 of its 52 references that are in the map. First posted on arXiv 5 Aug 2026.

Builds on

40 of 52 references.

Closest ancestors · 11

Same direction, newest first

  1. 2026GigaWorld-Policy-0.5: A Faster and Stronger WAM Empowered by AutoResearch15 Jul
  2. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr
  3. 2026World-Value-Action Model: Implicit Planning for Vision-Language-Action Systems16 Apr
  4. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
  5. 2026World Action Models are Zero-shot Policies17 Feb
  6. 2026VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model10 Feb
  7. 2026Causal World Modeling for Robot Control29 Jan
  8. 2026PointWorld: Scaling 3D World Models for In-The-Wild Robotic Manipulation7 Jan
  9. 2025Motus: A Unified Latent Action World Model15 Dec
  10. 2025WorldVLA: Towards Autoregressive Action World Model26 Jun
  11. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr

Other references · 29

  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
  5. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  6. 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
  7. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  8. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
  9. 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
  10. 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 626
  11. 2025SmolVLA: A Vision-Language-Action Model for Affordable and Efficient Roboticscited by 507
  12. 2025SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Modelcited by 502
17 more
  1. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
  2. 2025UniVLA: Learning to Act Anywhere with Task-centric Latent Actionscited by 386
  3. 2025X-VLA: Soft-Prompted Transformer as Scalable Cross-Embodiment Vision-Language-Action Modelcited by 285
  4. 2025π*0.6: a VLA That Learns From Experiencecited by 280
  5. 2025LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Modelscited by 278
  6. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  7. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Drivingcited by 127
  8. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
  9. 2025Spatial Forcing: Implicit Spatial Representation Alignment for Vision-language-action Modelcited by 125
  10. 2025VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Modelscited by 112
  11. 2026A Pragmatic VLA Foundation Modelcited by 97
  12. 2025GeoVLA: Empowering 3D Representations in Vision-Language-Action Modelscited by 68
  13. 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 50
  14. 2025DepthVLA: Enhancing Vision-Language-Action Models with Depth-Aware Spatial Reasoningcited by 44
  15. 2026Qwen-VLA: Unifying Vision-Language-Action Modeling across Tasks, Environments, and Robot Embodimentscited by 42
  16. 2026BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generationcited by 25
  17. 2026DreamWorld: Unified World Modeling in Video Generationcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Foresight Without Seeing: Latent Futures for World Action Models
  2. 2026GeoSem-WAM: Geometry- and Semantic-Aware World Action Models
  3. 2026Flex-π: A Multi-Stream World-Action Model with Compute Flexibility
  4. 2026DC-WAM: Dynamic-Centric Visual Supervision and Reasoning for World-Action Models
  5. 2026From World Models to World Action Models: Rethinking Next-State Prediction
  6. 2026XGenAct: Geometry-Enhanced World Action Models through Cross-Task Generation
  7. 2026Learning Visual Feature-Based World Models via Residual Latent Action
  8. 2026Spatially Aware World Action Model via Geometric Latent Diffusion