# HyWorldVLA: A Vision-Language-Action Model with Hybrid World Modeling for Autonomous Driving arXiv: https://arxiv.org/abs/2607.20988 · HTML: https://mlsanity.com/paper/2607.20988/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in World models for autonomous driving, one of 11 directions in World models. - Builds on 31 of its 57 references that are in the map. - First posted on arXiv 23 Jul 2026. ## Placed in - World models / [World models for autonomous driving](https://mlsanity.com/wm/world-models-for-autonomous-driving/) - Vision-language-action models / [VLA for autonomous driving](https://mlsanity.com/vla/vla-for-autonomous-driving/) - End-to-end autonomous driving / [Generative simulation for driving](https://mlsanity.com/driving/generative-simulation-for-driving/) ## Builds on (most cited first, 30 of 31) - [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022) - [π0: A Vision-Language-Action Flow Model for General Robot Control](https://mlsanity.com/paper/2410.24164/) (2024) - [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024) - [FAST: Efficient Action Tokenization for Vision-Language-Action Models](https://mlsanity.com/paper/2501.09747/) (2025) - [GAIA-1: A Generative World Model for Autonomous Driving](https://mlsanity.com/paper/2309.17080/) (2023) - [NuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles](https://mlsanity.com/paper/2106.11810/) (2021) - [NAVSIM: Data-Driven Non-Reactive Autonomous Vehicle Simulation and Benchmarking](https://mlsanity.com/paper/2406.15349/) (2024) - [AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning](https://mlsanity.com/paper/2506.13757/) (2025) - [Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillation](https://mlsanity.com/paper/2406.06978/) (2024) - [DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving](https://mlsanity.com/paper/2312.09245/) (2023) - [EMMA: End-to-End Multimodal Model for Autonomous Driving](https://mlsanity.com/paper/2410.23262/) (2024) - [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025) - [ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generation](https://mlsanity.com/paper/2503.19755/) (2025) - [OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Model](https://mlsanity.com/paper/2503.23463/) (2025) - [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025) - [Pseudo-Simulation for Autonomous Driving](https://mlsanity.com/paper/2506.04218/) (2025) - [DriveVLA-W0: World Models Amplify Data Scaling Law in Autonomous Driving](https://mlsanity.com/paper/2510.12796/) (2025) - [AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning](https://mlsanity.com/paper/2503.07608/) (2025) - [ADriver-I: A General World Model for Autonomous Driving](https://mlsanity.com/paper/2311.13549/) (2023) - [Doe-1: Closed-Loop Autonomous Driving with Large World Model](https://mlsanity.com/paper/2412.09627/) (2024) - [DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving](https://mlsanity.com/paper/2602.06521/) (2026) - [Vision-Language-Action Models for Autonomous Driving: Past, Present, and Future](https://mlsanity.com/paper/2512.16760/) (2025) - [Latent-WAM: Latent World Action Modeling for End-to-End Autonomous Driving](https://mlsanity.com/paper/2603.24581/) (2026) - [The Role of World Models in Shaping Autonomous Driving: A Comprehensive Survey](https://mlsanity.com/paper/2502.10498/) (2025) - [Large Motion Video Autoencoding with Cross-modal Video VAE](https://mlsanity.com/paper/2412.17805/) (2024) - [Latent Chain-of-Thought World Modeling for End-to-End Driving](https://mlsanity.com/paper/2512.10226/) (2025) - [ResWorld: Temporal Residual World Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2602.10884/) (2026) - [DreamerAD: Efficient Reinforcement Learning via Latent World Model for Autonomous Driving](https://mlsanity.com/paper/2603.24587/) (2026) - [ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving](https://mlsanity.com/paper/2604.02714/) (2026) - [CoWorld-VLA: Thinking in a Multi-Expert World Model for Autonomous Driving](https://mlsanity.com/paper/2605.10426/) (2026) ## Built on by (most cited first, 1 of 1) - [CoVLM-Bench: A Real-World Benchmark for Cooperative Driving Question Answering and Planning](https://mlsanity.com/paper/2609.35823/) (2026) ## Abstract (arXiv) Vision-Language-Action (VLA) models augmented with world modeling represent a promising paradigm for end-to-end autonomous driving. While pixel-level future prediction enables fine-grained spatiotemporal reasoning, it compromises robustness in noisy driving scenarios. Conversely, latent-based world models alleviate this sensitivity but often incur limited interpretability and representational degradation due to absent pixel-level grounding. To reconcile this trade-off, we propose HyWorldVLA, a hybrid world-VLA framework that unifies pixel-level supervision and latent representation learning. In the pre-training stage, HyWorldVLA predicts video latents encoded by a pre-trained video VAE, while simultaneously reconstructing video frames to provide precise pixel-level grounding. During the subsequent co-fine-tuning phase, the model exclusively predicts latent features, which are fed into an action expert to generate trajectories. Extensive experiments on NAVSIM v1 and v2 benchmarks demonstrate that HyWorldVLA significantly outperforms both pixel-based and latent-based world model baselines. Notably, we present the first comprehensive qualitative and quantitative analysis of world model noise robustness in autonomous driving, establishing a new benchmark for evaluating future architectures.