Making Foresight Actionable: Repurposing Representation Alignment in World Action Models
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World Action Models (WAMs) offer a promising route for robot manipulation by using video generation models to model future scene evolution before producing control actions. However, our empirical observations reveal a phenomenon: generating plausible visual futures does not always guarantee the extraction of accurate actions. To diagnose this failure, we conduct action-head attention analysis and causal interventions. We find that the action decoder fails to focus on task-relevant interaction regions and remains sensitive to perturbations in task-irrelevant areas. This reveals a representation mismatch: hidden states optimized for visual reconstruction are not inherently organized in a form useful for low-level action control. In this paper, we propose AGRA, an Action-Grounded Representation Alignment objective that regularizes the world-action interface by aligning intermediate video diffusion features with spatially coherent semantic representations from a foundation visual encoder. We evaluate AGRA on real-world manipulation tasks. Experiments show that AGRA makes world model representations more action-grounded: by focusing the action decoder on the correct interaction regions, it improves object localization accuracy and affordance understanding, and makes the policy more robust to perturbations in task-irrelevant regions. As a result, AGRA consistently improves both in-distribution performance and out-of-distribution generalization over the baseline world action model.
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 64 of its 66 references that are in the map. First posted on arXiv 10 Jun 2026.
Builds on
64 of 66 references.
Closest ancestors · 19
Same direction, newest first
- 2026Reconstruction or Semantics? What Makes a Latent Space Useful for Robotic World Models7 May
- 2026Motubrain: An Advanced World Action Model for Robot Control30 Apr
- 2026DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA31 Mar
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos6 Feb
- 2026Causal World Modeling for Robot Control29 Jan
- 2025Motus: A Unified Latent Action World Model15 Dec
- 2025GigaWorld-0: World Models as Data Engine to Empower Embodied AI25 Nov
- 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model31 Oct
- 2025Ctrl-World: A Controllable Generative World Model for Robot Manipulation11 Oct
- 2025RoboScape: Physics-informed Embodied World Model29 Jun
7 more
- 2025WorldVLA: Towards Autoregressive Action World Model26 Jun
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning11 Jun
- 2025WorldGym: World Model as An Environment for Policy Evaluation31 May
- 2025WorldEval: World Model as Real-World Robot Policies Evaluator25 May
- 2025FLARE: Robot Learning with Implicit World Modeling21 May
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models19 May
- 2023Diffusion Policy: Visuomotor Policy Learning via Action Diffusion7 Mar
Other references · 45
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,061
- 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 985
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 881
33 more
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2023Open X-Embodiment: Robotic Learning Datasets and RT-X Modelscited by 862
- 2024Genie: Generative Interactive Environmentscited by 747
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 658
- 2023Learning Universal Policies via Text-Guided Video Generationcited by 522
- 2024DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planningcited by 386
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 364
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 300
- 2025Diffusion Transformers with Representation Autoencoderscited by 295
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 289
- 2016Deep Visual Foresight for Planning Robot Motioncited by 285
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
- 2025EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Videocited by 211
- 2025Unified Video Action Modelcited by 205
- 2025Context as Memory: Scene-Consistent Interactive Long Video Generation with Memory Retrievalcited by 163
- 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 142
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 137
- 2023Denoising Diffusion Autoencoders are Unified Self-supervised Learnerscited by 130
- 2024Is Sora a World Simulator? A Comprehensive Survey on General World Models and Beyondcited by 111
- 2025VideoREPA: Learning Physics for Video Generation through Relational Alignment with Foundation Modelscited by 98
- 2023Diffusion Model as Representation Learnercited by 95
- 2025Video Generators are Robot Policiescited by 92
- 2024Deconstructing Denoising Diffusion Models for Self-Supervised Learningcited by 91
- 2025What matters for Representation Alignment: Global Information or Spatial Structure?cited by 84
- 2024The Matrix: Infinite-Horizon World Generation with Real-Time Moving Controlcited by 80
- 2025Vid2World: Crafting Video Diffusion Models to Interactive World Modelscited by 76
- 2026DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Controlcited by 76
- 2023Diffusion Models as Masked Autoencoderscited by 71
- 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 67
- 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 62
- 2026LDA-1B: Scaling Latent Dynamics Action Model via Universal Embodied Data Ingestioncited by 52
- 2025Cross-Frame Representation Alignment for Fine-Tuning Video Diffusion Modelscited by 12
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Informationcited by 2
- 2026Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Modelscited by 1
- 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulationcited by 1
- 2026MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generationcited by 0
- 2026SCULPT-VLA: Learning Structured Control through Staged Action Groundingcited by 0
- 2026The Right Future for Action: Learning Action-Relevant Predictive States in World Action Modelscited by 0
- 2026Beyond Visual Quality: A Study of Test-Time Planning with World Action Modelscited by 0
- 2026ActGaze: Learning Action-Grounded Gaze through Counterfactual Visual Interventions for High-Precision Manipulationcited by 0
- 2026Future Anchored Verification and Online Recovery for World Action Modelscited by 0
- 2026OpenWAM: An Open Framework for Composable World-Action Modelscited by 0
- 2026What Visual Representations Do World Action Models Need?cited by 0
- 2026What Visual Representations Do World Action Models Need?9 Oct 2026
- 2026OpenWAM: An Open Framework for Composable World-Action Models6 Oct 2026
- 2026Future Anchored Verification and Online Recovery for World Action Models5 Oct 2026
- 2026ActGaze: Learning Action-Grounded Gaze through Counterfactual Visual Interventions for High-Precision Manipulation24 Sep 2026
- 2026Beyond Visual Quality: A Study of Test-Time Planning with World Action Models21 Sep 2026
- 2026The Right Future for Action: Learning Action-Relevant Predictive States in World Action Models20 Sep 2026
- 2026SCULPT-VLA: Learning Structured Control through Staged Action Grounding20 Sep 2026
- 2026MT-WAM: Reorienting the One-Pass Predictive Representation Toward Action Generation18 Sep 2026
- 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation22 Aug 2026
- 2026Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Information7 Aug 2026
- 2026Action QFormer: Structured Representation Shaping under Action Supervision in Vision-Language-Action Models16 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026AIM: Intent-Aware Unified world action Modeling with Spatial Value Maps
- 2026What Limits Cross-Task Generalization in World Action Models?
- 2026RepWAM: World Action Modeling with Representation Visual-Action Tokenizers
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026PACT-WAM: Predicting Actions and Visual Foresight with Compact Temporal Encoding for Robot Manipulation
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026Event-Aligned Visual Action Reasoning for World Action Models
- 2026ST-WAM: Semantic-Temporal World Action Model for Robust Manipulation under Visual Distribution Shifts