HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics. Current WAMs generally follow two paradigms: the "Imagine-then-Execute" approach, which uses video prediction to infer actions via inverse dynamics, and the "Joint Modeling" approach, which jointly models actions and video representations. Based on systematic experiments, we observe a fundamental trade-off between these paradigms: the former explicitly leverages world models for generalizable transit but lacks interaction precision, whereas the latter enables fine-grained, temporally coherent action generation but is constrained by the exploration space of the training distribution. Motivated by these findings, we propose HarmoWAM, an end-to-end WAM that fully leverages a world model to unify predictive and reactive control, enabling both generalizable transit and precise manipulation. Specifically, the world model provides spatio-temporal physical priors that condition two complementary action experts: a predictive expert that leverages latent dynamics for iterative action generation, and a reactive expert that directly infers actions from predicted visual evolution. To enable adaptive coordination, a Process-Adaptive Gating Mechanism is proposed to automatically determine the timing and location of switching between them. This allows the world model to drive the reactive expert to expand the exploration space and the predictive expert to perform precise interactions across different stages of a task. For evaluation, we construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics. Notably, HarmoWAM achieves strong zero-shot generalization across these scenarios, significantly outperforming prior state-of-the-art VLA models and WAMs by margins of 33% and 29%, respectively.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 49 of its 61 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
49 of 61 references.
Closest ancestors · 14
Same direction, newest first
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Model18 Mar
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2026Causal World Modeling for Robot Control29 Jan
- 2026TC-IDM: Grounding Video Generation for Executable Zero-shot Robot Motion26 Jan
- 2025Act2Goal: From World Model To General Goal-conditioned Policy29 Dec
- 2025Motus: A Unified Latent Action World Model15 Dec
- 2025RynnVLA-002: A Unified Vision-Language-Action and World Model21 Nov
- 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model31 Oct
- 2025WorldVLA: Towards Autoregressive Action World Model26 Jun
- 2025FLARE: Robot Learning with Implicit World Modeling21 May
- 2025DreamGen: Unlocking Generalization in Robot Learning through Video World Models19 May
2 more
Other references · 35
- 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
- 2022RT-1: Robotics Transformer for Real-World Control at Scalecited by 2,007
- 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
- 2024Octo: An Open-Source Generalist Robot Policycited by 1,427
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2024DROID: A Large-Scale In-The-Wild Robot Manipulation Datasetcited by 801
- 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 670
- 2025FAST: Efficient Action Tokenization for Vision-Language-Action Modelscited by 643
23 more
- 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 411
- 2025AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systemscited by 390
- 2023Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulationcited by 353
- 2024GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulationcited by 290
- 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2025Unified Video Action Modelcited by 195
- 2025HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Modelcited by 176
- 2024RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulationcited by 166
- 2024Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulationcited by 142
- 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
- 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 127
- 2025OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoningcited by 105
- 2025Video Generators are Robot Policiescited by 86
- 2025WoW: Towards a World omniscient World model Through Embodied Interactioncited by 66
- 2025Vidar: Embodied Video Diffusion Model for Generalist Manipulationcited by 58
- 2024Diffusion-VLA: Generalizable and Interpretable Robot Foundation Model via Self-Generated Reasoningcited by 57
- 2025Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoningcited by 28
- 2026LaST0: Latent Spatio-Temporal Chain-of-Thought for Robotic Vision-Language-Action Modelcited by 28
- 2025AnyPos: Automated Task-Agnostic Actions for Bimanual Manipulationcited by 23
- 2025ManualVLA: A Unified VLA Model for Chain-of-Thought Manual Generation and Robotic Manipulationcited by 13
- 2025Video2Act: A Dual-System Video Diffusion Policy with Robotic Spatio-Motional Modelingcited by 13
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026World Action Models: A Surveycited by 10
- 2026RepWAM: World Action Modeling with Representation Visual-Action Tokenizerscited by 9
- 2026SANTS: A State-Adaptive Scheduler for World Action Modelscited by 7
- 2026StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulationcited by 3
- 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratiocited by 2
- 2026TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Trainingcited by 1
- 2026HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Modelscited by 1
- 2026DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imaginationcited by 0
- 2026Beyond Visual Quality: A Study of Test-Time Planning with World Action Modelscited by 0
- 2026DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinementcited by 0
- 2026DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinement21 Sep 2026
- 2026Beyond Visual Quality: A Study of Test-Time Planning with World Action Models21 Sep 2026
- 2026DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination8 Sep 2026
- 2026StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation11 Aug 2026
- 2026Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio9 Jul 2026
- 2026HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models5 Jul 2026
- 2026TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Training3 Jul 2026
- 2026World Action Models: A Survey18 Jun 2026
- 2026RepWAM: World Action Modeling with Representation Visual-Action Tokenizers11 Jun 2026
- 2026SANTS: A State-Adaptive Scheduler for World Action Models27 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026GE-Act 2.0: Pretraining and Scaling a World-Action Model for Robotic Manipulation
- 2026World Models for Robotic Manipulation: A Survey
- 2026DELE-w0.5: Inferring Action from Future Latent State for Robotic Manipulation
- 2026τ0-WM: A Unified Video-Action World Model for Robotic Manipulation
- 2026InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data
- 2026OpenWAM: An Open Framework for Composable World-Action Models
- 2026Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination
- 2026World-Action Models for Robot Learning and Control: A Survey