# HarmoWAM: Harmonizing Generalizable and Precise Manipulation via Adaptive World Action Models

arXiv: https://arxiv.org/abs/2605.10942 · HTML: https://mlsanity.com/paper/2605.10942/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 49 of its 61 references that are in the map.
- First posted on arXiv 11 May 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)

## Builds on (most cited first, 30 of 49)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [OpenVLA: An Open-Source Vision-Language-Action Model](https://mlsanity.com/paper/2406.09246/) (2024)
- [Qwen3-VL Technical Report](https://mlsanity.com/paper/2511.21631/) (2025)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [RT-1: Robotics Transformer for Real-World Control at Scale](https://mlsanity.com/paper/2212.06817/) (2022)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [Octo: An Open-Source Generalist Robot Policy](https://mlsanity.com/paper/2405.12213/) (2024)
- [Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Success](https://mlsanity.com/paper/2502.19645/) (2025)
- [DROID: A Large-Scale In-The-Wild Robot Manipulation Dataset](https://mlsanity.com/paper/2403.12945/) (2024)
- [RDT-1B: a Diffusion Foundation Model for Bimanual Manipulation](https://mlsanity.com/paper/2410.07864/) (2024)
- [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulation](https://mlsanity.com/paper/2411.19650/) (2024)
- [AgiBot World Colosseo: A Large-scale Manipulation Platform for Scalable and Intelligent Embodied Systems](https://mlsanity.com/paper/2503.06669/) (2025)
- [WorldVLA: Towards Autoregressive Action World Model](https://mlsanity.com/paper/2506.21539/) (2025)
- [Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation](https://mlsanity.com/paper/2312.13139/) (2023)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning](https://mlsanity.com/paper/2601.16163/) (2026)
- [GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation](https://mlsanity.com/paper/2410.06158/) (2024)
- [Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations](https://mlsanity.com/paper/2412.14803/) (2024)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Unified Video Action Model](https://mlsanity.com/paper/2503.00200/) (2025)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [HybridVLA: Collaborative Diffusion and Autoregression in a Unified Vision-Language-Action Model](https://mlsanity.com/paper/2503.10631/) (2025)
- [DreamGen: Unlocking Generalization in Robot Learning through Video World Models](https://mlsanity.com/paper/2505.12705/) (2025)
- [RoboMIND: Benchmark on Multi-embodiment Intelligence Normative Data for Robot Manipulation](https://mlsanity.com/paper/2412.13877/) (2024)
- [Gen2Act: Human Video Generation in Novel Scenarios enables Generalizable Robot Manipulation](https://mlsanity.com/paper/2409.16283/) (2024)

## Built on by (most cited first, 10 of 10)

- [RepWAM: World Action Modeling with Representation Visual-Action Tokenizers](https://mlsanity.com/paper/2606.13674/) (2026)
- [World Action Models: A Survey](https://mlsanity.com/paper/2606.20781/) (2026)
- [SANTS: A State-Adaptive Scheduler for World Action Models](https://mlsanity.com/paper/2605.27947/) (2026)
- [Understanding and Mitigating the Video-Action Generalization Gap via Temporal Ratio](https://mlsanity.com/paper/2607.08127/) (2026)
- [StageWAM: Joint-Embedding Stage Prediction for World-Action Models in Robot Manipulation](https://mlsanity.com/paper/2608.10780/) (2026)
- [TACO: TActile World Model as a Self-COrrector for Scalable Robot Policy Post-Training](https://mlsanity.com/paper/2607.02840/) (2026)
- [HALO-WA: Hybrid-Attention Latent-Guided Online Reinforcement Learning for World-Action Models](https://mlsanity.com/paper/2607.04265/) (2026)
- [DeCAL: Towards Physically-Grounded Dexterous Vision-Language-Action Models via Contact-Aware Latent Co-Imagination](https://mlsanity.com/paper/2609.09119/) (2026)
- [Beyond Visual Quality: A Study of Test-Time Planning with World Action Models](https://mlsanity.com/paper/2609.24745/) (2026)
- [DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinement](https://mlsanity.com/paper/2609.24868/) (2026)

## Abstract (arXiv)

World Action Models (WAMs) have emerged as a promising paradigm for robot control by modeling physical dynamics. Current WAMs generally follow two paradigms: the "Imagine-then-Execute" approach, which uses video prediction to infer actions via inverse dynamics, and the "Joint Modeling" approach, which jointly models actions and video representations. Based on systematic experiments, we observe a fundamental trade-off between these paradigms: the former explicitly leverages world models for generalizable transit but lacks interaction precision, whereas the latter enables fine-grained, temporally coherent action generation but is constrained by the exploration space of the training distribution. Motivated by these findings, we propose HarmoWAM, an end-to-end WAM that fully leverages a world model to unify predictive and reactive control, enabling both generalizable transit and precise manipulation. Specifically, the world model provides spatio-temporal physical priors that condition two complementary action experts: a predictive expert that leverages latent dynamics for iterative action generation, and a reactive expert that directly infers actions from predicted visual evolution. To enable adaptive coordination, a Process-Adaptive Gating Mechanism is proposed to automatically determine the timing and location of switching between them. This allows the world model to drive the reactive expert to expand the exploration space and the predictive expert to perform precise interactions across different stages of a task. For evaluation, we construct three training-unseen test environments across six real-world robotic tasks, covering variations in background, position, and object semantics. Notably, HarmoWAM achieves strong zero-shot generalization across these scenarios, significantly outperforming prior state-of-the-art VLA models and WAMs by margins of 33% and 29%, respectively.