mlsanityData to 14 Oct 2026

AWM-VLA: AlignedWorld Modeling for Efficient and Explainable Vision-Language-Action Policies

arXiv 2609.27753 ↗First posted 15 Aug 2026cs.CVcited by 0

On the map: World models → Embodied and robot world models

also in 2 more
Abstract (arXiv)

Vision-language-action (VLA) models have become a powerful paradigm for generalist robotic manipulation, yet they are often reactive: the policy maps the current observation directly to an action chunk without reasoning about the long-term consequences of its decisions. Prior attempts to endow policies with world models either reconstruct future frames in pixel space---expensive and dominated by task-irrelevant detail---or decouple the world model from the policy, weakening control. We present AWM-VLA, a unified framework that embeds aligned world modeling directly inside a diffusion-transformer policy. Following the Future Latent REpresentation Alignment (FLARE) principle, we add learnable future tokens whose intermediate activations are aligned with vision-language embeddings of future observations, enabling the policy to anticipate long-term consequences while generating actions. We extend this paradigm in two ways. First, we introduce an object-centric decoupled alignment objective that predicts future object-level semantics alongside the global future embedding, improving both interpretability and multi-instruction generalization. Second, we balance the global and object-centric alignment terms against the action flow-matching loss through a principled weighting, yielding a controllable accuracy--interpretability trade-off. On RoboCasa and humanoid tabletop manipulation benchmarks, AWM-VLA outperforms prior VLA and world-model baselines by up to 21% in success rate, improves generalization to novel objects and instructions, and produces object-centric rationales that are preferred by human raters in 83 of cases. Our approach adds only a few learnable tokens to the policy and is compatible with any diffusion or flow-matching policy, making aligned world modeling an inexpensive, broadly applicable component of generalist manipulation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 36 of its 42 references that are in the map. First posted on arXiv 15 Aug 2026.

Builds on

36 of 42 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025FLARE: Robot Learning with Implicit World Modeling21 May
  2. 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar

Other references · 34

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  8. 2022Scalable Diffusion Models with Transformerscited by 6,054
  9. 2022Flow Matching for Generative Modelingcited by 5,702
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  11. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  12. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
22 more
  1. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  2. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  3. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  4. 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
  5. 2023Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardwarecited by 1,568
  6. 2018World Modelscited by 1,513
  7. 2018Learning Latent Dynamics for Planning from Pixelscited by 1,264
  8. 2023Mastering Diverse Domains through World Modelscited by 1,253
  9. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  10. 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 852
  11. 2017A Survey on Multi-Task Learningcited by 840
  12. 2018Multi-Task Learning as Multi-Objective Optimizationcited by 821
  13. 2022A Generalist Agentcited by 808
  14. 2018End-to-End Multi-Task Learning with Attentioncited by 534
  15. 2024RoboCasa: Large-Scale Simulation of Everyday Tasks for Generalist Robotscited by 407
  16. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  17. 2022Behavior Transformers: Cloning k modes with one stonecited by 304
  18. 2021Bridge Data: Boosting Generalization of Robotic Skills with Cross-Domain Datasetscited by 250
  19. 2020Language Conditioned Imitation Learning over Unstructured Datacited by 209
  20. 20243D Diffusion Policy: Generalizable Visuomotor Policy Learning via Simple 3D Representationscited by 116
  21. 2025UniHOI: Unified Human-Object Interaction Understanding via Unified Token Spacecited by 10
  22. 2026MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonalitycited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026WAVE-0: Bridging VLA Policies and WAMs with Lightweight Latent Future Prediction
  2. 2026DeltaVLA: Prior-Guided Vision-Language-Action Models via World Knowledge Variation
  3. 2026Scaling World Model for Hierarchical Manipulation Policies
  4. 2025Goal-VLA: Image-Generative VLMs as Object-Centric World Models Empowering Zero-shot Robot Manipulation
  5. 2025Grounding Actions in Camera Space: Observation-Centric Vision-Language-Action Policy
  6. 2026ObjDream: Guiding Vision-Language-Action Models with Object-Level Understanding and Future State Prediction
  7. 2026PHR-VLA: Planning Horizon Reasoning for Vision-Language-Action Models
  8. 2026SLIP-VLA: Single-Step Latent Imagination for Policy Learning in Vision-Language-Action Models