CausalDrive: Real-time Causal World Models for Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
World models have emerged as a promising paradigm for scaling autonomous driving (AD) data, yet existing video generative models fall short as interactive simulators. Layout-conditioned renderers rely on "oracle" future trajectories of all background agents, rendering them strictly non-reactive. Conversely, pure action-conditioned predictors lack semantic control over complex interactions and suffer from prohibitive diffusion latencies, hindering closed-loop policy learning. To bridge this gap, we present CausalDrive, a controllable, real-time foundation driving world renderer. CausalDrive operates solely on the initial front-view frame, the ego-vehicle's trajectory, and a macroscopic text prompt. By excluding future NPC layouts, we compel the model to intrinsically predict causal interactions, enabling text-driven control over Driving Sociology, allowing users to dynamically orchestrate diverse counterfactual reactions to identical ego-actions. To overcome the efficiency bottleneck and address the covariate shift in autoregressive generation, we propose a novel Context-Forced DMD architecture. This combines continuous flow-matching with a self-correcting distillation objective, achieving interactive speeds of 12 FPS. This breakthrough transforms the passive video generator into a playable neural simulator. We demonstrate its versatility across three downstream applications: (1) generative closed-loop evaluation with significantly mitigated collision artifacts, (2) large-scale Reinforcement Learning (RL) post-training driven by a Video2Reward module, and (3) real-time human-in-the-loop simulation. Extensive experiments validate that policies trained within CausalDrive's reactive scenarios exhibit superior interaction capabilities in the real world.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 39 of its 41 references that are in the map. First posted on arXiv 13 Jun 2026.
Builds on
39 of 41 references.
Closest ancestors · 11
Same direction, newest first
- 2025AD-R1: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving with Impartial World Models25 Nov
- 2025Orbis: Overcoming Challenges of Long-Horizon Prediction in Driving World Models17 Jul
- 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
- 2024DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation11 Mar
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2021MetaDrive: Composing Diverse Driving Scenarios for Generalizable Reinforcement Learning26 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
- 2017CARLA: An Open Urban Driving Simulator10 Nov
Other references · 28
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
- 2022Planning-oriented Autonomous Drivingcited by 1,087
- 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 864
- 2025Self Forcing: Bridging the Train-Test Gap in Autoregressive Video Diffusioncited by 625
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 354
- 2022BEVFusion: A Simple and Robust LiDAR-Camera Fusion Frameworkcited by 336
16 more
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 334
- 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
- 2023MagicDrive: Street View Generation with Diverse 3D Geometry Controlcited by 252
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 252
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 198
- 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 168
- 2024UniScene: Unified Occupancy-centric Driving Scene Generationcited by 106
- 2024DriveArena: A Closed-loop Generative Simulation Platform for Autonomous Drivingcited by 98
- 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 92
- 2024GEM: A Generalizable Ego-Vision Multimodal World Model for Fine-Grained Ego-Motion, Object Dynamics, and Scene Composition Controlcited by 78
- 2025DriveDPO: Policy Learning via Safety DPO For End-to-End Autonomous Drivingcited by 39
- 2025SimScale: Learning to Drive via Real-World Simulation at Scalecited by 36
- 2024DrivingSphere: Building a High-fidelity 4D World for Closed-loop Simulationcited by 34
- 2025WorldLens: Full-Spectrum Evaluations of Driving World Models in Real Worldcited by 23
- 2024OLiDM: Object-aware LiDAR Diffusion Models for Autonomous Drivingcited by 13
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Modelscited by 1
- 2026BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generationcited by 0
- 2026CASCADE: A Spatio-Temporal-Causal Reasoning Representation and Dataset for Drivingcited by 0
- 2026ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulationcited by 0
- 2026CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Drivingcited by 0
- 2026SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models9 Oct 2026
- 2026CoDrive: Cross-Vehicle World-Consistent Video Generation with Precise Trajectory Control for Cooperative Driving28 Sep 2026
- 2026ZYT-World: A Real-Time Controllable World Model for Closed-Loop Autonomous-Driving Simulation18 Sep 2026
- 2026CASCADE: A Spatio-Temporal-Causal Reasoning Representation and Dataset for Driving7 Sep 2026
- 2026BehaviorWorldGen: Closing the Loop between Action Models and World Simulators via Controllable Behavior-Aware Structured World Generation23 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026ChainFlow-VLA: Causal Flow Planning with Vision-Language Models
- 2026DrivingGen: A Comprehensive Benchmark for Generative Video World Models in Autonomous Driving
- 2026CausalDriveBench: Evaluating Causal Reasoning in Vision-Language-Action Models for Autonomous Driving
- 2026Causality-Aware End-to-End Autonomous Driving via Ego-Centric Joint Scene Modeling
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT
- 2023DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving
- 2026Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
- 2026Orbis 2: A Hierarchical World Model for Driving