mlsanityData to 14 Oct 2026

CtrlWAM: Controllable World Action Models with Aligned Intent and Foresight

arXiv 2610.00859 ↗First posted 1 Oct 2026cs.CVcited by 0

On the map: World models → Embodied and robot world models

Abstract (arXiv)

World action models (WAMs) jointly predict actions (intent) and visual future (foresight). Standard training adds noise to recorded actions and video simultaneously, but such training paradigms introduce a mismatch: perturbed actions imply counterfactual future visual, while the noised video remains tied to the GT recording. In low-noise regime, the scene geometry and even the dynamic behavior remain clearly visible from the noisy future frames despite the added noise. We present CtrlWAM, which executes perturbed actions in a simulator and pairs them with their noised visual consequences for joint WAM learning. To accommodate the different denoising requirements of video and actions, we introduce warped video--action noise schedules that aim to keep visual layout responsive as action predictions evolve. We further extend the action interface from ego-only control to a variable number of agent streams, allowing a unified model to represent predicted or commanded futures for multiple agents. Driving experiments show more accurate action forecasts, closer agreement between generated video and actions, and better following of supplied commands; robotics experiments show stronger motion fidelity and controllability. Matched controls support the benefit of off-path renders for command following and manipulation fidelity. Together, these findings contribute to a more controllable world action model. Project page: https://ctrl-wam.github.io/

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and robot world models, one of 11 directions in World models. Builds on 27 of its 32 references that are in the map. First posted on arXiv 1 Oct 2026.

Builds on

27 of 32 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026DIDO: Distilling Interaction-Centric Dynamics into One-Step Denoising for World Action Models14 Sep
  2. 2026Unified 4D World Action Modeling from Video Priors with Asynchronous Denoising29 Apr
  3. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?17 Mar
  4. 2026World Action Models are Zero-shot Policies17 Feb
  5. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets3 Apr

Other references · 22

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  7. 2024Genie: Generative Interactive Environmentscited by 747
  8. 2018Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networkscited by 690
  9. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
  10. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  11. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  12. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 450
10 more
  1. 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllabilitycited by 347
  2. 2023EvalCrafter: Benchmarking and Evaluating Large Video Generation Modelscited by 340
  3. 2016Deep Visual Foresight for Planning Robot Motioncited by 285
  4. 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
  5. 2025Difix3D+: Improving 3D Reconstructions with Single-Step Diffusion Modelscited by 161
  6. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 156
  7. 2026WorldArena: A Unified Benchmark for Evaluating Perception and Functional Utility of Embodied World Modelscited by 76
  8. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 64
  9. 2025EWMBench: Evaluating Scene, Motion, and Semantic Quality in Embodied World Modelscited by 50
  10. 2024Beyond FVD: Enhanced Evaluation Metrics for Video Generation Qualitycited by 22

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Foresight Without Seeing: Latent Futures for World Action Models
  2. 2026DualWAM: Dual-System World Action Models for Asynchronous Global Planning and Local Refinement
  3. 2026Efficient-WAM: A 1B-Parameter World-Action Model with Low-Cost Future Imagination
  4. 2026NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
  5. 2026RAP-WAM: Learning World-Action Models through Predictive Rehearsal and Action Perturbations
  6. 2026World Action Modeling with Progressive Visual Planning
  7. 2026MARS: Systematic Multi-Level Acceleration for World Action Models
  8. 2026DeltaWAM: Distilling Action Tangent Fields into World Action Models