Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation
On the map: World models → World models for autonomous driving
also in 1 more
Abstract (arXiv)
World action models~(WAMs) have shown great promise for autonomous driving and urban navigation. Built upon Vision-Language-Action models or video generation models, existing approaches suffer key limitations: (1) High inference latency due to future observation prediction at test time, and (2) tightly coupled video and action modeling leading to representational mismatch and degraded generalization. To address both issues, we propose Metis, an end-to-end WAM framework that decouples video generation and action prediction. Specifically, Metis employs a Mixture-of-Transformers architecture with dedicated experts for video generation and action prediction, preserving the intrinsic distributional properties of each task. To enhance efficiency, we introduce an asymmetric attention mask that enables joint training of both experts while allowing the action model to bypass explicit video generation during inference. This design ensures training-inference consistency and significantly reduces computational costs without compromising planning performance. Extensive experiments demonstrate state-of-the-art performance on the NAVSIM navhard and navtest benchmarks and the CityWalker navigation benchmark, validating both the generalizability and efficiency across diverse tasks. Real-robot deployments further confirm the practical feasibility of our approach.
Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 94 of its 98 references that are in the map. First posted on arXiv 14 Jun 2026.
Builds on
94 of 98 references.
Closest ancestors · 21
Same direction, newest first
- 2026Uni-World VLA: Interleaved World Modeling and Planning for Autonomous Driving28 Mar
- 2026Vega: Learning to Drive with Natural Language Instructions26 Mar
- 2026Bridging Scene Generation and Planning: Driving with World Model via Unifying Vision and Motion Representation16 Mar
- 2025DriveLaW:Unifying Planning and Video Generation in a Latent Driving World29 Dec
- 2025WorldRFT: Latent World Model Planning with Reinforcement Fine-Tuning for Autonomous Driving22 Dec
- 2025DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving14 Oct
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
- 2025Epona: Autoregressive Diffusion World Model for Autonomous Driving30 Jun
- 2025ReSim: Reliable World Simulation for Autonomous Driving11 Jun
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2025GAIA-2: A Controllable Multi-View Generative World Model for Autonomous Driving26 Mar
9 more
- 2025MiLA: Multi-view Intensive-fidelity Long-term Video Generation World Model for Autonomous Driving20 Mar
- 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation24 Jan
- 2024DrivingWorld: Constructing World Model for Autonomous Driving via Video GPT27 Dec
- 2024DrivingGPT: Unifying Driving World Modeling and Planning with Multi-modal Autoregressive Transformers24 Dec
- 2024Doe-1: Closed-Loop Autonomous Driving with Large World Model12 Dec
- 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
- 2024Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability27 May
- 2024GenAD: Generalized Predictive Model for Autonomous Driving14 Mar
- 2023Driving into the Future: Multiview Visual Forecasting and Planning with World Model for Autonomous Driving29 Nov
Other references · 73
- 2023GPT-4 Technical Reportcited by 18,132
- 2023Visual Instruction Tuningcited by 7,792
- 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
- 2024Genie: Generative Interactive Environmentscited by 706
- 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
- 2022TransFuser: Imitation with Transformer-Based Sensor Fusion for Autonomous Drivingcited by 477
- 2024Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Modelscited by 467
- 2026World Action Models are Zero-shot Policiescited by 411
- 2025WorldVLA: Towards Autoregressive Action World Modelcited by 351
- 2024NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarkingcited by 343
38 more
- 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
- 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 311
- 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
- 2026Causal World Modeling for Robot Controlcited by 289
- 2026Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planningcited by 274
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2023Parting with Misconceptions about Learning-based Vehicle Motion Planningcited by 238
- 2023NoMaD: Goal Masked Diffusion Policies for Navigation and Explorationcited by 234
- 2025Motus: A Unified Latent Action World Modelcited by 231
- 2024NaVid: Video-based VLM Plans the Next Step for Vision-and-Language Navigationcited by 222
- 2023ViNT: A Foundation Model for Visual Navigationcited by 210
- 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
- 2025World Simulation with Video Foundation Models for Physical AIcited by 183
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
- 2023Panacea: Panoramic and Controllable Video Generation for Autonomous Drivingcited by 165
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
- 2022GNM: A General Navigation Model to Drive Any Robotcited by 161
- 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156
- 2025Pseudo-Simulation for Autonomous Drivingcited by 141
- 2025Alpamayo-R1: Bridging Reasoning and Action Prediction for Generalizable Autonomous Driving in the Long Tailcited by 136
- 2025mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAscited by 133
- 2025Genie Envisioner: A Unified World Foundation Platform for Robotic Manipulationcited by 129
- 2025StreamVLN: Streaming Vision-and-Language Navigation via SlowFast Context Modelingcited by 107
- 2020BADGR: An Autonomous Self-Supervised Learning-Based Navigation Systemcited by 97
- 2024NavGPT-2: Unleashing Navigational Reasoning Capability for Large Vision-Language Modelscited by 96
- 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
- 2025Hydra-MDP++: Advancing End-to-End Driving via Expert-Guided Hydra-Distillationcited by 93
- 2025DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planningcited by 91
- 2024MagicDrive-V2: High-Resolution Long Video Generation for Autonomous Driving with Adaptive Controlcited by 90
- 2024DRAMA: An Efficient End-to-end Motion Planner for Autonomous Driving with Mambacited by 87
- 2025F1: A Vision-Language-Action Model Bridging Understanding and Generation to Actionscited by 85
- 2025ARTEMIS: Autoregressive End-to-End Trajectory Planning with Mixture of Experts for Autonomous Drivingcited by 66
- 2025Embodied Navigation Foundation Modelcited by 65
- 2025Generalized Trajectory Scoring for End-to-end Multimodal Planningcited by 63
- 2025AdaThinkDrive: Adaptive Thinking via Reinforcement Learning for Autonomous Drivingcited by 57
- 2025Ground Slow, Move Fast: A Dual-System Foundation Model for Generalizable Vision-and-Language Navigationcited by 56
- 2022ViKiNG: Vision-Based Kilometer-Scale Navigation with Geographic Hintscited by 55
- 2026InternVLA-A1: Unifying Understanding, Generation and Action for Robotic Manipulationcited by 48
50 most cited of 73 All 73 (JSON)
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Drivingcited by 5
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulationcited by 2
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigationcited by 2
- 20264D-WAM: 4D Consistent World Modeling for Autonomous Drivingcited by 1
- 2026SUV: Future Scene Understanding as Video Generation for End-to-End Drivingcited by 0
- 2026MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Drivingcited by 0
- 2026DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Modelscited by 0
- 2026ReDrive: Shaping Representations with World Modeling for End-to-End Drivingcited by 0
- 2026V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Drivingcited by 0
- 2026Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imaginationcited by 0
- 2026Sparse-WAM: Accelerating World Action Models via Action-Guided Sparse Imagination30 Sep 2026
- 2026V2X-WAM: A Cooperative World Action Model for End-to-End Autonomous Driving29 Sep 2026
- 2026ReDrive: Shaping Representations with World Modeling for End-to-End Driving27 Sep 2026
- 2026DriveReferee: Geometric Safety Verdicts Need Not Be Learned for Driving World-Action Models19 Sep 2026
- 2026MM-Future: Multi-Mode Joint World-Action Modeling for Autonomous Driving17 Sep 2026
- 20264D-WAM: 4D Consistent World Modeling for Autonomous Driving10 Aug 2026
- 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Driving7 Aug 2026
- 2026UniNav: A Unified World-Action Diffusion Model for Visual Navigation4 Aug 2026
- 2026SUV: Future Scene Understanding as Video Generation for End-to-End Driving4 Aug 2026
- 2026N0-TWAM: Scaling Tactile-Native World-Action Model for Contact-Rich Manipulation26 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026SV-WAM: An Efficient Surround-View World-Action Model for End-to-End Autonomous Driving
- 2026ReWorld: Representation Learning for World Action Models
- 2026HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving
- 2026Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving
- 2026DriveWAM: Video Generative Priors Enable Scalable World-Action Modeling for Autonomous Driving
- 2025ProphetDWM: A Driving World Model for Rolling Out Future Actions and Videos
- 2026World-Action Models for Robot Learning and Control: A Survey
- 2026BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving