# Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving

arXiv: https://arxiv.org/abs/2603.24581 · HTML: https://mlsanity.com/paper/2603.24581/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in World models for autonomous driving, one of 11 directions in World models.
- Builds on 47 of its 48 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Placed in

- World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/)
- End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/)

## Builds on (most cited first, 30 of 47)

- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023)
- [VAD: Vectorized Scene Representation for Efficient Autonomous Driving](https://mlsanity.com/paper/2303.12077/) (2023)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021)
- [TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Driving](https://mlsanity.com/paper/2205.15997/) (2022)
- [Multi-Modal Fusion Transformer for End-to-End Autonomous Driving](https://mlsanity.com/paper/2104.09224/) (2021)
- [NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking](https://mlsanity.com/paper/2406.15349/) (2024)
- [Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability](https://mlsanity.com/paper/2405.17398/) (2024)
- [DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2411.15139/) (2024)
- [Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving](https://mlsanity.com/paper/2311.17918/) (2023)
- [VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planning](https://mlsanity.com/paper/2402.13243/) (2024)
- [DriveDreamer: Towards Real-world-driven World Models for Autonomous Driving](https://mlsanity.com/paper/2309.09777/) (2023)
- [SparseDrive: End-to-End Autonomous Driving via Sparse Scene Representation](https://mlsanity.com/paper/2405.19620/) (2024)
- [OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving](https://mlsanity.com/paper/2311.16038/) (2023)
- [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025)
- [DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation](https://mlsanity.com/paper/2403.06845/) (2024)
- [PandaSet: Advanced Sensor Suite Dataset for Autonomous Driving](https://mlsanity.com/paper/2112.12610/) (2021)
- [Enhancing End-to-End Autonomous Driving with Latent World Model](https://mlsanity.com/paper/2406.08481/) (2024)
- [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025)
- [DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving](https://mlsanity.com/paper/2510.12796/) (2025)
- [VLP: Vision Language Planning for Autonomous Driving](https://mlsanity.com/paper/2401.05577/) (2024)
- [DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Driving](https://mlsanity.com/paper/2503.07656/) (2025)
- [Epona: Autoregressive Diffusion World Model for Autonomous Driving](https://mlsanity.com/paper/2506.24113/) (2025)
- [Copilot4D: Learning Unsupervised World Models for Autonomous Driving via Discrete Diffusion](https://mlsanity.com/paper/2311.01017/) (2023)
- [UniScene: Unified Occupancy-centric Driving Scene Generation](https://mlsanity.com/paper/2412.05435/) (2024)

## Built on by (most cited first, 30 of 33)

- [DriveFuture: Future-Aware Latent World Models for Autonomous Driving](https://mlsanity.com/paper/2605.09701/) (2026)
- [The DAWN of World-Action Interactive Models](https://mlsanity.com/paper/2605.11550/) (2026)
- [GraphWorld: Long-Horizon Planning with World Models for End-to-End Autonomous Driving](https://mlsanity.com/paper/2606.16274/) (2026)
- [WA-JEPA: Rethinking the Video JEPA Paradigm for World-Action Modeling in Autonomous Driving](https://mlsanity.com/paper/2608.20974/) (2026)
- [Human Cognition in Machines: A Unified Perspective of World Models](https://mlsanity.com/paper/2604.16592/) (2026)
- [DA-WAM: Decision-Aligned Future Latents for Driving World Models](https://mlsanity.com/paper/2608.19085/) (2026)
- [Discrete-WAM: Unified Discrete Vision-Action Token Editing for World-Policy Learning](https://mlsanity.com/paper/2606.05645/) (2026)
- [RISE: Adaptive Imagination for World Action Models](https://mlsanity.com/paper/2608.20430/) (2026)
- [ReWorld: Representation Learning for World Action Models](https://mlsanity.com/paper/2606.27504/) (2026)
- [ForgeDrive: Bidirectional Cross-Conditioning for Unified Visual-Action Generation in Autonomous Driving](https://mlsanity.com/paper/2606.31226/) (2026)
- [HiDrive: A Closed-Loop Benchmark for High-Level Autonomous Driving](https://mlsanity.com/paper/2605.09972/) (2026)
- [HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving](https://mlsanity.com/paper/2607.20988/) (2026)
- [MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving](https://mlsanity.com/paper/2609.20377/) (2026)
- [Sometimes You Gotta Run Before You Can Walk: Run-then-Walk Scheduling Strategy for VLM Autonomous Driving](https://mlsanity.com/paper/2609.25831/) (2026)
- [SimVTA: Simulation-Grounded Video-Trajectory Alignment for Driving World Action Models](https://mlsanity.com/paper/2610.13781/) (2026)
- [Not All History Helps: Velocity-Aware Selective Memory for Long-Horizon End-to-End Autonomous Driving](https://mlsanity.com/paper/2608.15573/) (2026)
- [READ: Learning Risk-Informed Fields for End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.12371/) (2026)
- [Relationally Grounded Latent World Models for Autonomous Driving](https://mlsanity.com/paper/2609.24626/) (2026)
- [ForeDrive: Foresight-Guided End-to-End Autonomous Driving with a Planning-Relevant Latent World Model](https://mlsanity.com/paper/2609.26299/) (2026)
- [SlackDrive: Reclaiming Runtime Slack for Adaptive Driving Inference](https://mlsanity.com/paper/2609.28064/) (2026)
- [MomWorld: Momentum-Aware Latent World Model for Long-Horizon Autonomous Driving](https://mlsanity.com/paper/2609.33737/) (2026)
- [ReDrive: Shaping Representations with World Modeling for End-to-End Driving](https://mlsanity.com/paper/2609.33854/) (2026)
- [AD-E2E-JEPA: A Joint-Embedding Predictive Architecture For End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.34085/) (2026)
- [RoXDrive: Closed-Loop Reinforcement Learning for End-to-End Autonomous Driving via Action-Faithful Rollouts](https://mlsanity.com/paper/2609.36851/) (2026)
- [V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2609.37098/) (2026)
- [ReWAM: Reciprocal World Action Models for Interactive Autonomous Driving](https://mlsanity.com/paper/2609.39245/) (2026)
- [KineWorld: Action-Induced Transport Fields for Embodied World Modeling](https://mlsanity.com/paper/2610.06349/) (2026)
- [Do Better Visual Representations Always Lead to Better End-to-End Autonomous Driving?](https://mlsanity.com/paper/2610.09695/) (2026)
- [PlanWAM: Planning-Shaped Future Representations for End-to-End Autonomous Driving](https://mlsanity.com/paper/2610.11382/) (2026)
- [FPD-Drive: Future Policy Distillation for End-to-End Autonomous Driving](https://mlsanity.com/paper/2610.13232/) (2026)

## Abstract (arXiv)

We introduce Latent-WAM, an efficient end-to-end autonomous driving framework that achieves strong trajectory planning through spatially-aware and dynamics-informed latent world representations. Existing world-model-based planners suffer from inadequately compressed representations, limited spatial understanding, and underutilized temporal dynamics, resulting in sub-optimal planning under constrained data and compute budgets. Latent-WAM addresses these limitations with two core modules: a Spatial-Aware Compressive World Encoder (SCWE) that distills geometric knowledge from a foundation model and compresses multi-view images into compact scene tokens via learnable queries, and a Dynamic Latent World Model (DLWM) that employs a causal Transformer to autoregressively predict future world status conditioned on historical visual and motion representations. Extensive experiments on NAVSIM v2 and HUGSIM demonstrate new state-of-the-art results: 89.3 EPDMS on NAVSIM v2 and 28.9 HD-Score on HUGSIM, surpassing the best prior perception-free method by 3.2 EPDMS with significantly less training data and a compact 104M-parameter model.