# Vid2WAM: Distilling Video Diffusion Priors into World Action Models

arXiv: https://arxiv.org/abs/2608.08558 · HTML: https://mlsanity.com/paper/2608.08558/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Embodied and robot world models, one of 11 directions in World models.
- Builds on 25 of its 45 references that are in the map.
- First posted on arXiv 9 Aug 2026.

## Placed in

- World models / [Embodied and robot world models](https://mlsanity.com/wm/embodied-and-robot-world-models/)
- Diffusion models / [Diffusion and flow policies for robotics](https://mlsanity.com/diffusion/diffusion-and-flow-policies-for-robotics/)

## Builds on (most cited first, 25 of 25)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024)
- [π0.5: a Vision-Language-Action Model with Open-World Generalization](https://mlsanity.com/paper/2504.16054/) (2025)
- [RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulation](https://mlsanity.com/paper/2506.18088/) (2025)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Orca: Progressive Learning from Complex Explanation Traces of GPT-4](https://mlsanity.com/paper/2306.02707/) (2023)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [LIBERO-Plus: In-depth Robustness Analysis of Vision-Language-Action Models](https://mlsanity.com/paper/2510.13626/) (2025)
- [GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation](https://mlsanity.com/paper/2410.06158/) (2024)
- [Motus: A Unified Latent Action World Model](https://mlsanity.com/paper/2512.13030/) (2025)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Unified Video Action Model](https://mlsanity.com/paper/2503.00200/) (2025)
- [RoboDreamer: Learning Compositional World Models for Robot Imagination](https://mlsanity.com/paper/2404.12377/) (2024)
- [GenAug: Retargeting behaviors to unseen situations via Generative Augmentation](https://mlsanity.com/paper/2302.06671/) (2023)
- [Large Video Planner Enables Generalizable Robot Control](https://mlsanity.com/paper/2512.15840/) (2025)
- [τ0-WM: A Unified Video-Action World Model for Robotic Manipulation](https://mlsanity.com/paper/2606.01027/) (2026)
- [AMPLIFY: Actionless Motion Priors for Robot Learning from Videos](https://mlsanity.com/paper/2506.14198/) (2025)
- [Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination](https://mlsanity.com/paper/2606.10040/) (2026)
- [S-VAM: Shortcut Video-Action Model by Self-Distilling Geometric and Semantic Foresight](https://mlsanity.com/paper/2603.16195/) (2026)
- [EVA: Aligning Video World Models with Executable Robot Actions via Inverse Dynamics Rewards](https://mlsanity.com/paper/2603.17808/) (2026)
- [RoboCurate: Harnessing Diversity with Action-Verified Neural Trajectory for Robot Learning](https://mlsanity.com/paper/2602.18742/) (2026)
- [Privileged Foresight Distillation: Zero-Cost Future Correction for World Action Models](https://mlsanity.com/paper/2604.25859/) (2026)
- [CKT-WAM: Parameter-Efficient Context Knowledge Transfer Between World Action Models](https://mlsanity.com/paper/2605.06247/) (2026)
- [Supervise What Survives: Geometry-Guided VLA Adaptation from Synthetic Robot Videos](https://mlsanity.com/paper/2606.24448/) (2026)

## Built on by (most cited first, 2 of 2)

- [EVO-WAM: Evolving World Action Models through Video-Action Verification](https://mlsanity.com/paper/2609.38057/) (2026)
- [OpenWAM: An Open Framework for Composable World-Action Models](https://mlsanity.com/paper/2610.07922/) (2026)

## Abstract (arXiv)

World Action Models (WAMs) improve robot policy learning by jointly modeling future visual dynamics and actions. However, their scalability and generalization remain constrained by their reliance on costly expert demonstrations. We challenge this by asking whether future supervision for WAMs must originate from target-task expert trajectories. In this paper, we propose Vid2WAM, an offline distillation framework that transfers visual diffusion priors from a large video foundation model into a compact WAM student. Given an observation and language instruction, Vid2WAM distills supervision through two complementary channels: task-conditioned future rollouts directly supervise the student's future prediction branch, while an inverse dynamics model recovers embodiment-specific pseudo-actions for action learning. To robustly integrate synthetic and real supervision, we introduce source-aware residual action adaptation that learns source-specific corrections around a shared action backbone and mitigates interference from noisy pseudo-actions. During inference, both the video teacher and inverse dynamics model are discarded, leaving only the WAM student for efficient deployment. Simulation and real-world experiments demonstrate that Vid2WAM improves novel-task generalization and data efficiency under limited expert demonstrations while preserving low-latency inference.