Drive-HWM: Hierarchical World Models for Dynamic-Latent Guided Autonomous Driving
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
World models offer a promising paradigm for autonomous driving by predicting how traffic scenes may evolve and using such predictions to support action generation. However, existing approaches either separate future prediction from action generation or jointly predict them at the same temporal scale, making it difficult to simultaneously achieve long-horizon anticipation and responsive, observation-grounded decision making. We present Drive-HWM, a hierarchical slow--fast world modeling framework that organizes future representation prediction and action generation at complementary temporal scales. The slow world model predicts multi-step future representations to capture extended scene evolution. To explicitly model the abundant motion dynamics in driving environments, we introduce Dynamic-Aware Latents learned through optical-flow prediction. Guided by these future representations, the fast model uses a lightweight multimodal backbone and an autoregressive expert to jointly predict the next frame and the immediate action from the latest observation. Next-frame prediction encourages the fast model to capture imminent scene evolution, while one-step action generation allows decisions to be continuously updated as new observations arrive. Extensive experiments on NAVSIM v1 and v2 demonstrate the strong driving performance of Drive-HWM. Comprehensive ablation studies further validate the effectiveness of the hierarchical slow--fast design, dynamics-aware future representations, and joint next-frame and action prediction.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 53 of its 59 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
53 of 59 references.
Closest ancestors · 10
Same direction, newest first
- 2025Rethinking Driving World Model as Synthetic Data Generator for Perception Tasks22 Oct
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos24 May
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024DriveWorld: 4D Pre-trained Scene Understanding via World Models for Autonomous Driving7 May
- 2023Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion2 Nov
- 2023GAIA-1: A Generative World Model for Autonomous Driving29 Sep
- 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles22 Jun
Other references · 43
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
- 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Modelcited by 455
- 2021Multi-Modal Fusion Transformer for End-to-End Autonomous Drivingcited by 372
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
31 more
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2025Motus: A Unified Latent Action World Modelcited by 231
- 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 172
- 2023JARVIS-1: Open-World Multi-task Agents with Memory-Augmented Multimodal Language Modelscited by 170
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
- 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 149
- 2025Pseudo-Simulation for Autonomous Drivingcited by 141
- 2026GigaWorld-Policy: An Efficient Action-Centered World--Action Modelcited by 129
- 2025Unified Vision-Language-Action Modelcited by 128
- 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Modelcited by 91
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
- 2025DriveMoE: Mixture-of-Experts for Vision-Language-Action Model in End-to-End Autonomous Drivingcited by 85
- 2024NavCoT: Boosting LLM-Based Vision-and-Language Navigation via Learning Disentangled Reasoningcited by 77
- 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 74
- 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 68
- 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 66
- 2025DiffVLA: Vision-Language Guided Diffusion Planning for Autonomous Drivingcited by 64
- 2025VL-JEPA: Joint Embedding Predictive Architecture for Vision-languagecited by 61
- 2024CarLLaVA: Vision language models for camera-only closed-loop drivingcited by 57
- 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 57
- 2025World-in-World: World Models in a Closed-Loop Worldcited by 44
- 2025Pre-Trained Video Generative Models as World Simulatorscited by 43
- 2025IRL-VLA: Training an Vision-Language-Action Policy via Reward World Modelcited by 31
- 2025SafeAuto: Knowledge-Enhanced Safe Autonomous Driving with Multimodal Foundation Modelscited by 22
- 2024FASIONAD : FAst and Slow FusION Thinking Systems for Human-Like Autonomous Driving with Adaptive Feedbackcited by 18
- 2025DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Modelscited by 12
- 2025TS-VLM: Text-Guided SoftSort Pooling for Vision-Language Models in Multi-View Driving Reasoningcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- 2025A Survey of World Models for Autonomous Driving
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
- 2026ReWorld: Representation Learning for World Action Models
- 2026Orbis 2: A Hierarchical World Model for Driving
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving
- 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
- 2026DA-WAM: Decision-Aligned Future Latents for Driving World Models