# Driver-WM: A Driver-Centric Traffic-Conditioned Latent World Model for In-Cabin Dynamics Rollout

arXiv: https://arxiv.org/abs/2605.05092 · HTML: https://mlsanity.com/paper/2605.05092/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 49 of its 66 references that are in the map.
- First posted on arXiv 6 May 2026.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)

## Builds on (most cited first, 30 of 49)

- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Control](https://mlsanity.com/paper/2307.15818/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/1801.07455/) (2018)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [DriveLM: Driving with Graph Visual Question Answering](https://mlsanity.com/paper/2312.14150/) (2023)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability](https://mlsanity.com/paper/2405.17398/) (2024)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [MotionBERT: A Unified Perspective on Learning Human Motion Representations](https://mlsanity.com/paper/2210.06551/) (2022)
- [DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation](https://mlsanity.com/paper/2403.06845/) (2024)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation](https://mlsanity.com/paper/2410.13571/) (2024)
- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model](https://mlsanity.com/paper/2507.00603/) (2025)
- [Milestones in Autonomous Driving and Intelligent Vehicles: Survey of Surveys](https://mlsanity.com/paper/2303.17220/) (2023)
- [3D and 4D World Modeling: A Survey](https://mlsanity.com/paper/2509.07996/) (2025)
- [TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild](https://mlsanity.com/paper/2104.04029/) (2021)
- [AIDE: A Vision-Driven Multi-View, Multi-Modal, Multi-Tasking Dataset for Assistive Driving Perception](https://mlsanity.com/paper/2307.13933/) (2023)
- [Hybrid Latent Reasoning via Reinforcement Learning](https://mlsanity.com/paper/2505.18454/) (2025)
- [Brain4Cars: Car That Knows Before You Do via Sensory-Fusion Deep Learning Architecture](https://mlsanity.com/paper/1601.00740/) (2016)
- [Social Interactions for Autonomous Driving: A Review and Perspectives](https://mlsanity.com/paper/2208.07541/) (2022)
- [DriveVA: Video Action Models are Zero-Shot Drivers](https://mlsanity.com/paper/2604.04198/) (2026)
- [UniDrive-WM: Unified Understanding, Planning and Generation World Model for Autonomous Driving](https://mlsanity.com/paper/2601.04453/) (2026)
- [Hybrid-Prediction Integrated Planning for Autonomous Driving](https://mlsanity.com/paper/2402.02426/) (2024)

## Built on by (most cited first, 6 of 6)

- [Risk-Aware Selective Multimodal Driver Monitoring with Driver-State World Modeling](https://mlsanity.com/paper/2606.26922/) (2026)
- [PV-WM: A Heterogeneous Micro-Macro World Model for Articulated Pedestrian-Vehicle Co-Rollout](https://mlsanity.com/paper/2609.07328/) (2026)
- [DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting](https://mlsanity.com/paper/2609.08117/) (2026)
- [MOJITO: Modal Joint Learning for Unified End-to-End Autonomous Driving](https://mlsanity.com/paper/2607.23511/) (2026)
- [TriDrive: Joint Driver, Vehicle, and Road Modeling for Forecasting and Driver Monitoring](https://mlsanity.com/paper/2609.33000/) (2026)
- [MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving](https://mlsanity.com/paper/2609.33737/) (2026)

## Abstract (arXiv)

Safe L2/L3 driving automation requires anticipating human-in-the-loop reactions during shared-control transitions. While most driving world models forecast the external environment, in-cabin intelligence remains strictly recognition-oriented and lacks multi-step rollout capabilities for driver dynamics. We introduce Driver-WM, a driver-centric latent world model that rolls out in-cabin dynamics causally conditioned on out-cabin traffic context. This formulation unifies physical kinematics forecasting with auxiliary behavioral and emotional semantic recognition. Operating in a compact latent space constructed from frozen vision-language features, Driver-WM adopts a dual-stream architecture to separately encode external traffic and internal driver states. These streams are directionally coupled via a gated causal injection mechanism, which uses a learned vector gate to modulate external contextual perturbations while strictly enforcing temporal causality. Experiments on AIDE show robust long-horizon forecasting on reactive high-motion clips, improved driver/traffic semantic alignment, and controlled interventions that expose the external-to-internal mechanism.