CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight
On the map: World models → Embodied and robot world models
Abstract (arXiv)
World action models (WAMs) jointly predict actions (intent) and visual future (foresight). Standard training adds noise to recorded actions and video simultaneously, but such training paradigms introduce a mismatch: perturbed actions imply counterfactual future visual, while the noised video remains tied to the GT recording. In low-noise regime, the scene geometry and even the dynamic behavior remain clearly visible from the noisy future frames despite the added noise. We present CtrlWAM, which executes perturbed actions in a simulator and pairs them with their noised visual consequences for joint WAM learning. To accommodate the different denoising requirements of video and actions, we introduce warped video--action noise schedules that aim to keep visual layout responsive as action predictions evolve. We further extend the action interface from ego-only control to a variable number of agent streams, allowing a unified model to represent predicted or commanded futures for multiple agents. Driving experiments show more accurate action forecasts, closer agreement between generated video and actions, and better following of supplied commands; robotics experiments show stronger motion fidelity and controllability. Matched controls support the benefit of off-path renders for command following and manipulation fidelity. Together, these findings contribute to a more controllable world action model. Project page: https://ctrl-wam.github.io/
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 27 of its 32 references that are in the map. First posted on arXiv 1 Oct 2026.
Builds on
27 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2026DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models14 Sep
- 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
- 2026World Action Models are Zero-shot Policies17 Feb
- 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr
Other references · 22
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024Genie: Generative Interactive Environmentscited by 747
- 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
- 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
- 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
10 more
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
- 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
- 2016Deep Visual Foresight for Planning Robot Motioncited by 285
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
- 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
- 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
- 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 64
- 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 50
- 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 22
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Foresight Without Seeing: Latent Futures for World Action Models
- 2026DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinement
- 2026Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination
- 2026NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
- 2026RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
- 2026World Action Modeling with Progressive Visual Planning
- 2026MARS: Systematic Multi-Level Acceleration for World Action Models
- 2026DeltaWAM: Distilling Action Tangent Fields into World Action Models