mlsanityData to 7 Oct 2026

DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving

arXiv 2605.28544 ↗First posted 27 May 2026cs.CVcited by 12

On the map: World models → World models for autonomous driving

also in 2 more
Abstract (arXiv)

Pretrained foundation models have become an important basis for end-to-end autonomous driving. In contrast to vision-language models pretrained primarily on static image-text pairs, video generative models capture temporal dynamics and motion priors that are naturally suited for driving. We present DriveWAM, a driving world-action model that adapts a pretrained video diffusion transformer into an autoregressive video-action policy. DriveWAM organizes video and action streams into a unified temporal token sequence and trains them under a joint flow-matching objective, preserving the pretrained video-generation architecture while adapting its large-scale video priors to action generation. To incorporate high-level scene understanding, we introduce scene-evolving driving guidance, where a frozen VLM produces chunk-specific semantic intent to guide video-action generation. To keep long-horizon rollout bounded, we further introduce selective KV memory, which maintains bounded modality-aware video and action memory pools through relevance-redundancy cache selection at inference time. Experiments on NAVSIM and the PhysicalAI-Autonomous-Vehicles benchmark show that DriveWAM achieves strong planning performance, and a data-scaling study from 4k to 100k driving clips further confirms the scaling potential of world-action modeling for end-to-end autonomous driving.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 58 of its 63 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

58 of 63 references.

Closest ancestors · 9

Same direction, newest first

  1. 2026X-World: Controllable Ego-Centric Multi-Camera World Models for Scalable End-to-End Driving20 Mar
  2. 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation16 Mar
  3. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  4. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  5. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  6. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  7. 2025VaViM and VaVAM: Autonomous Driving through Video Generative Modeling21 Feb
  8. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  9. 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar

Other references · 49

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2022Flow Matching for Generative Modelingcited by 5,426
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  9. 2025Qwen3-VL Technical Reportcited by 2,534
  10. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  11. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
  12. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
37 more
  1. 2022Planning-oriented Autonomous Drivingcited by 1,068
  2. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  3. 2024PaliGemma: A versatile 3B VLM for transfercited by 679
  4. 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 634
  5. 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 569
  6. 2023DriveLM: Driving with Graph Visual Question Answeringcited by 527
  7. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  8. 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
  9. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
  10. 2026World Action Models are Zero-shot Policiescited by 411
  11. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  12. 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
  13. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  14. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  15. 2026Causal World Modeling for Robot Controlcited by 289
  16. 2024Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representationscited by 279
  17. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  18. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
  19. 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
  20. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Drivingcited by 233
  21. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 204
  22. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  23. 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
  24. 2023A Language Agent for Autonomous Drivingcited by 161
  25. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
  26. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
  27. 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
  28. 2024FIFO-Diffusion: Generating Infinite Videos from Text without Trainingcited by 134
  29. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 129
  30. 2022MoVQ: Modulating Quantized Vectors for High-Fidelity Image Generationcited by 125
  31. 2024Towards learning-based planning:The nuPlan benchmark for real-world autonomous drivingcited by 103
  32. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 100
  33. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationcited by 100
  34. 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 85
  35. 2025LangCoop: Collaborative Driving with Languagecited by 25
  36. 2026Flow caching for autoregressive video generationcited by 19
  37. 2026DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planningcited by 17

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 12.

12 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
  2. 2026UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation
  3. 2026DriveVA: Video Action Models are Zero-Shot Drivers
  4. 2026FlashDrive: Flash Vision-Language-Action Inference for Autonomous Driving
  5. 2026BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving
  6. 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
  7. 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving
  8. 2026ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving