# UNIVERSE: Unified Video Action Models for Autonomous Driving with Flexible Mask-Modulated Modality Generation

arXiv: https://arxiv.org/abs/2607.05133 · HTML: https://mlsanity.com/paper/2607.05133/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 34 of its 67 references that are in the map.
- First posted on arXiv 6 Jul 2026.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)

## Builds on (most cited first, 30 of 34)

- [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025)
- [World Models](https://mlsanity.com/paper/1803.10122/) (2018)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [World Action Models are Zero-shot Policies](https://mlsanity.com/paper/2602.15922/) (2026)
- [Fast-WAM: Do World Action Models Need Test-time Future Imagination?](https://mlsanity.com/paper/2603.16666/) (2026)
- [Causal World Modeling for Robot Control](https://mlsanity.com/paper/2601.21998/) (2026)
- [Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning](https://mlsanity.com/paper/2601.16163/) (2026)
- [MagicDrive: Street View Generation with Diverse 3D Geometry Control](https://mlsanity.com/paper/2310.02601/) (2023)
- [Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets](https://mlsanity.com/paper/2504.02792/) (2025)
- [Unified Video Action Model](https://mlsanity.com/paper/2503.00200/) (2025)
- [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025)
- [GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2503.20523/) (2025)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [End-to-End Driving with Online Trajectory Evaluation via BEV World Model](https://mlsanity.com/paper/2504.01941/) (2025)
- [mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs](https://mlsanity.com/paper/2512.15692/) (2025)
- [GigaWorld-Policy: An Efficient Action-Centered World--Action Model](https://mlsanity.com/paper/2603.17240/) (2026)
- [DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving](https://mlsanity.com/paper/2510.12796/) (2025)
- [Epona: Autoregressive Diffusion World Model for Autonomous Driving](https://mlsanity.com/paper/2506.24113/) (2025)
- [Video Generators are Robot Policies](https://mlsanity.com/paper/2508.00795/) (2025)
- [Large Video Planner Enables Generalizable Robot Control](https://mlsanity.com/paper/2512.15840/) (2025)
- [DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT](https://mlsanity.com/paper/2412.19505/) (2024)
- [VideoVLA: Video Generators Can Be Generalizable Robot Manipulators](https://mlsanity.com/paper/2512.06963/) (2025)
- [Doe-1: Closed-Loop Autonomous Driving with Large World Model](https://mlsanity.com/paper/2412.09627/) (2024)
- [DriveLaW:Unifying Planning and Video Generation in a Latent Driving World](https://mlsanity.com/paper/2512.23421/) (2025)
- [ReSim: Reliable World Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.09981/) (2025)
- [Dual-Stream Diffusion for World-Model Augmented Vision-Language-Action Model](https://mlsanity.com/paper/2510.27607/) (2025)
- [OmniNWM: Omniscient Driving Navigation World Models](https://mlsanity.com/paper/2510.18313/) (2025)
- [From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction](https://mlsanity.com/paper/2510.19654/) (2025)
- [ImagiDrive: A Unified Imagination-and-Planning Framework for Autonomous Driving](https://mlsanity.com/paper/2508.11428/) (2025)
- [DriveVA: Video Action Models are Zero-Shot Drivers](https://mlsanity.com/paper/2604.04198/) (2026)

## Built on by (most cited first, 9 of 9)

- [MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving](https://mlsanity.com/paper/2609.20377/) (2026)
- [Binding Multiple Modalities via Multimodal Wasserstein Barycenter](https://mlsanity.com/paper/2609.33800/) (2026)
- [SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models](https://mlsanity.com/paper/2610.13781/) (2026)
- [DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models](https://mlsanity.com/paper/2609.22762/) (2026)
- [Honeycomb: Constant-Size Scene Memory Representation for Video World Models](https://mlsanity.com/paper/2609.37690/) (2026)
- [Artemis: Geometry-Grounded Multi-Agent Driving World Models with Shared 3D State and Progressive Memory Update](https://mlsanity.com/paper/2610.07031/) (2026)
- [MESSENGER: Memory-Enhanced Sequential Scene Flow Estimation via Autoregressive Next-Frame Forecasting](https://mlsanity.com/paper/2610.10759/) (2026)
- [MemReg: Streaming Outdoor LiDAR Point Cloud Registration with Hybrid Memory Buffers](https://mlsanity.com/paper/2610.12624/) (2026)
- [DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving](https://mlsanity.com/paper/2610.17903/) (2026)

## Abstract (arXiv)

World Action Models (WAMs) have shown strong potential for improving action generalization in autonomous driving by using future video prediction as dense supervision for scene dynamics and temporal causality. However, it remains unclear which architecture better transfers video-modeling benefits to trajectory generation. Existing cascaded or dual-DiT designs separate video imagination from action prediction, weakening the transfer of video-learned world dynamics to the trajectory branch: the action model may still overfit dataset-specific driving priors, while the video model only indirectly regularizes planning. We propose UNIVERSE, a unified video-action model built upon a single mask-modulated Diffusion Transformer. By co-training future video latents and ego-trajectory tokens within shared generative parameters, UNIVERSE allows dense video supervision to directly shape trajectory denoising, leading to stronger cross-domain action generalization. To ensure causal validity and efficient deployment, we introduce a Modality-Decoupling Visibility Mask, which shares historical context across modalities while blocking mutual attention between future video and trajectory tokens. This prevents future-target leakage and enables trajectory-only inference by removing future-video denoising at test time, achieving a 4.3× speedup over joint video-action rollout while maintaining comparable planning accuracy. The same model also supports video-only and joint video-action rollouts. Experiments show that UNIVERSE achieves 91.0 PDMS on NAVSIM (vs. 89.6 for the Two-DiT variant), and demonstrates strong zero-shot transfer to nuScenes and Bench2Drive without fine-tuning, while ablations confirm the importance of single-DiT unification, video co-training, and mask-based modality decoupling.