DeepSight: Long-Horizon World Modeling via Latent States Prediction for End-to-End Autonomous Driving
On the map: World models → World models for autonomous driving
also in 3 more
Abstract (arXiv)
End-to-end autonomous driving systems are increasingly integrating Vision-Language Model (VLM) architectures, incorporating text reasoning or visual reasoning to enhance the robustness and accuracy of driving decisions. However, the reasoning mechanisms employed in most methods are direct adaptations from general domains, lacking in-depth exploration tailored to autonomous driving scenarios, particularly within visual reasoning modules. In this paper, we propose a driving world model that performs parallel prediction of latent semantic features for consecutive future frames in the bird's-eye-view (BEV) space, thereby enabling long-horizon modeling of future world states. We also introduce an efficient and adaptive text reasoning mechanism that utilizes additional social knowledge and reasoning capabilities to further improve driving performance in challenging long-tail scenarios. We present a novel, efficient, and effective approach that achieves state-of-the-art (SOTA) results on the closed-loop Bench2drive benchmark. Codes are available at: https://github.com/hotdogcheesewhite/DeepSight.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in World models for autonomous driving, one of 11 directions in World models. Builds on 32 of its 40 references that are in the map. First posted on arXiv 11 May 2026.
Builds on
32 of 40 references.
Closest ancestors · 7
Same direction, newest first
- 2025World4Drive: End-to-End Autonomous Driving via Intention-aware Physical Latent World Model1 Jul
- 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
- 2025End-to-End Driving with Online Trajectory Evaluation via BEV World Model2 Apr
- 2025HERMES: A Unified Self-Driving World Model for Simultaneous 3D Scene Understanding and Generation24 Jan
- 2024DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation17 Oct
- 2024Navigation-Guided Sparse Scene Representation for End-to-End Autonomous Driving26 Sep
- 2024Enhancing End-to-End Autonomous Driving with Latent World Model12 Jun
Other references · 25
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,485
- 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
- 2022Planning-oriented Autonomous Drivingcited by 1,068
- 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
- 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuningcited by 252
- 2024Bench2Drive: Towards Multi-Ability Benchmarking of Closed-Loop End-To-End Autonomous Drivingcited by 250
- 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
- 2024Hydra-MDP: End-to-end Multimodal Planning with Multi-target Hydra-Distillationcited by 244
- 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
- 2025ORION: A Holistic End-to-End Autonomous Driving Framework by Vision-Language Instructed Action Generationcited by 177
- 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 168
13 more
- 2025FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Drivingcited by 165
- 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 142
- 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 125
- 2025GoalFlow: Goal-Driven Flow Matching for Multimodal Trajectories Generation in End-to-End Autonomous Drivingcited by 117
- 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
- 2024ChatScene: Knowledge-Enabled Safety-Critical Scenario Generation for Autonomous Vehiclescited by 92
- 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
- 2025Think Twice: Enhancing LLM Reasoning by Scaling Multi-round Test-time Thinkingcited by 46
- 2025Centaur: Robust End-to-End Autonomous Driving with Test-Time Trainingcited by 41
- 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
- 2025Context-Alignment: Activating and Enhancing LLM Capabilities in Time Seriescited by 34
- 2025ReasonPlan: Unified Scene Prediction and Decision Reasoning for Closed-loop Autonomous Drivingcited by 22
- 2023Pre-training on Synthetic Driving Data for Trajectory Predictioncited by 18
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
- 2026Learning Vision-Language-Action World Models for Autonomous Driving
- 2026Xiaomi OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation
- 2026Latent World Models for Automated Driving: A Unified Taxonomy, Evaluation Framework, and Open Challenges
- 2026WorldVLM: Combining World Model Forecasting and Vision-Language Reasoning
- 2026DriveWorld-VLA: Unified Latent-Space World Modeling with Vision-Language-Action for Autonomous Driving
- 2026EponaV2: Driving World Model with Comprehensive Future Reasoning
- 2026X-Mind: Efficient Visual Chain-of-Thought via Predictive World Model for End-to-End Driving