mlsanityData to 7 Oct 2026

UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation

arXiv 2607.05133 ↗First posted 6 Jul 2026cs.CVcited by 5

On the map: World models → World models for autonomous driving

Abstract (arXiv)

World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers video-modeling benefits to trajectory generation. Existing cascaded or dual-DiT designs separate video imagination from action prediction, weakening the transfer of video-learned world dynamics to the trajectory branch: the action model may still overfit dataset-specific driving priors, while the video model only indirectly regularizes planning. We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Diffusion Transformer. By co-training future video latents and ego-trajectory tokens within shared generative parameters, UNIVERSE allows dense video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual attention between future video and trajectory tokens. This prevents future-target leakage and enables trajectory-only inference by removing future-video denoising at test time, achieving a 4.3× speedup over joint video-action rollout while maintaining comparable planning accuracy. The same model also supports video-only and joint video-action rollouts. Experiments show that UNIVERSE achieves 91.0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification, video co-training, and mask-based modality decoupling.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 34 of its 67 references that are in the map. First posted on arXiv 6 Jul 2026.

Builds on

34 of 67 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026DriveVA: Video Action Models are Zero-Shot Drivers5 Apr
  2. 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
  3. 2025OmniNWM: Omniscient Driving Navigation World Models21 Oct
  4. 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
  5. 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
  6. 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
  7. 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos24 May
  8. 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
  9. 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
  10. 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving20 Mar
  11. 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
  12. 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
2 more
  1. 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
  2. 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep

Other references · 20

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  2. 2018World Modelscited by 1,444
  3. 2026World Action Models are Zero-shot Policiescited by 411
  4. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
  5. 2026Causal World Modeling for Robot Controlcited by 289
  6. 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
  7. 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 250
  8. 2025Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasetscited by 218
  9. 2025Unified Video Action Modelcited by 195
  10. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  11. 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
  12. 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 129
8 more
  1. 2025Video Generators are Robot Policiescited by 86
  2. 2025Large Video Planner Enables Generalizable Robot Controlcited by 67
  3. 2025VideoVLA: Video Generators Can Be Generalizable Robot Manipulatorscited by 61
  4. 2025Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Modelcited by 33
  5. 2025From Forecasting to Planning: Policy World Model for Collaborative State-Action Predictioncited by 29
  6. 2025ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Drivingcited by 24
  7. 2025Genesis: Multimodal Driving Scene Generation with Spatio-Temporal and Cross-Modal Consistencycited by 21
  8. 2025CoGen: 3D Consistent Video Generation via Adaptive Conditioning for Autonomous Drivingcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
  2. 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
  3. 2025MaskGWM: A Generalizable Driving World Model with Video Mask Reconstruction
  4. 2026ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving
  5. 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
  6. 2026Adaptive-WAM: Quality-Guided Early-Exit Planning from Intermediate Video-Diffusion Features
  7. 2026ReWorld: Representation Learning for World Action Models
  8. 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving