mlsanityData to 14 Oct 2026

CogDriver: Integrating Cognitive Inertia for Temporally Coherent Planning in Autonomous Driving

arXiv 2509.00789 ↗First posted 31 Aug 2025cs.CVcited by 5

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

The pursuit of autonomous agents capable of temporally coherent planning is hindered by a fundamental flaw in current vision-language models (VLMs): they lack cognitive inertia. Operating on isolated snapshots, these models cannot form a continuous understanding of the environment, leading to erratic decision jitter and a failure to execute complex, multi-step maneuvers. To remedy this, we introduce CogDriver, a framework designed to build a stable internal representation by instilling this crucial cognitive property. Our work makes two key contributions: (1) We present CogDriver-Data, a large-scale vision-language-action dataset whose narrative annotations provide the supervisory signal for learning temporal dynamics and persistent intent. (2) We develop the CogDriver-Agent, an architecture featuring a sparse temporal memory to maintain a stable internal state. This is enabled by a spatiotemporal knowledge distillation approach that explicitly teaches decision coherence. Comprehensive experiments validate our paradigm: CogDriver-Agent achieves a 22% increase in the closed-loop Driving Score on Bench2Drive and a 21% reduction in mean L2 error on nuScenes, establishing a new state-of-the-art. These significant gains in both long-term decision-making and imitation accuracy provide strong evidence that our agent successfully maintains a temporally coherent internal state, bridging the gap toward more reliable autonomous driving. Project link: https://ocean-luna.github.io/CogDriver.github.io/.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 50 of its 51 references that are in the map. First posted on arXiv 31 Aug 2025.

Builds on

50 of 51 references.

Closest ancestors · 12

Same direction, newest first

  1. 2025OmniDrive: A Holistic Vision-Language Dataset for Autonomous Driving with Counterfactual Reasoning6 Apr
  2. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Driving29 Oct
  3. 2024CarLLaVA: Vision language models for camera-only closed-loop driving14 Jun
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
  5. 2024RAG-Driver: Generalisable Driving Explanations with Retrieval-Augmented In-Context Learning in Multi-Modal Large Language Model16 Feb
  6. 2024GPT-4V as Traffic Assistant: An In-depth Look at Vision Language Model on Complex Traffic Events3 Feb
  7. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  8. 2023LingoQA: Visual Question Answering for Autonomous Driving21 Dec
  9. 2023DriveMLM: Aligning Multi-Modal Large Language Models with Behavioral Planning States for Autonomous Driving14 Dec
  10. 2023Reason2Drive: Towards Interpretable and Chain-based Reasoning for Autonomous Driving6 Dec
  11. 2023Vision Language Models in Autonomous Driving: A Survey and Outlook22 Oct
  12. 2023DriveGPT4: Interpretable End-to-end Autonomous Driving via Large Language Model2 Oct

Other references · 38

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  5. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  6. 2024Qwen2 Technical Reportcited by 2,120
  7. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  8. 2022Planning-oriented Autonomous Drivingcited by 1,087
  9. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 622
  10. 2024LLaVA-NeXT-Interleave: Tackling Multi-image, Video, and 3D in Large Multimodal Modelscited by 584
  11. 2023EVA-02: A Visual Representation for Neon Genesiscited by 381
  12. 2022ST-P3: End-to-end Vision-based Autonomous Driving via Spatial-Temporal Feature Learningcited by 368
26 more
  1. 2023GPT-Driver: Learning to Drive with GPTcited by 347
  2. 2024VADv2: End-to-End Vectorized Autonomous Driving via Probabilistic Planningcited by 289
  3. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 248
  4. 2023Is Ego Status All You Need for Open-Loop End-to-End Autonomous Driving?cited by 242
  5. 2023Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Drivingcited by 232
  6. 2018Textual Explanations for Self-Driving Vehiclescited by 226
  7. 2023Rethinking the Open-Loop Evaluation of End-to-End Autonomous Driving in nuScenescited by 191
  8. 2023Think Twice before Driving: Towards Scalable Decoders for End-to-End Autonomous Drivingcited by 169
  9. 2023A Language Agent for Autonomous Drivingcited by 164
  10. 2023MotionLM: Multi-Agent Motion Forecasting as Language Modelingcited by 156
  11. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
  12. 2019Talk2Car: Taking Control of Your Self-Driving Carcited by 142
  13. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 126
  14. 2024VLP: Vision Language Planning for Autonomous Drivingcited by 126
  15. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  16. 2025Don't Shake the Wheel: Momentum-Aware Planning in End-to-End Autonomous Drivingcited by 84
  17. 2024CoVLA: Comprehensive Vision-Language-Action Dataset for Autonomous Drivingcited by 77
  18. 2024A Survey on Autonomous Driving Datasets: Statistics, Annotation Quality, and a Future Outlookcited by 66
  19. 2023Rank2Tell: A Multimodal Driving Dataset for Joint Importance Ranking and Reasoningcited by 63
  20. 2023From task structures to world models: What do LLMs know?cited by 46
  21. 2024DME-Driver: Integrating Human Decision Logic and 3D Scene Perception in Autonomous Drivingcited by 43
  22. 2024WorldGPT: Empowering LLM as Multimodal World Modelcited by 42
  23. 2023Open-sourced Data Ecosystem in Autonomous Driving: the Present and Futurecited by 23
  24. 2025ReAL-AD: Towards Human-Like Reasoning in End-to-End Autonomous Drivingcited by 15
  25. 2025World Models for Cognitive Agents: Transforming Edge Intelligence in Future Networkscited by 12
  26. 2025CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Drivingcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving
  2. 2025ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
  3. 2026BrainWAM: Action-Space Coordination of Semantic Priors and Predictive Dynamics for Autonomous Driving
  4. 2026LaST-VLA: Thinking in Latent Spatio-Temporal Space for Vision-Language-Action in Autonomous Driving
  5. 2026Cognitive Dual-Process Planning for Autonomous Driving with Structured Scene Knowledge and Verifiable Reasoning-Action Consistency
  6. 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning
  7. 2026SGDrive: Scene-to-Goal Hierarchical World Cognition for Autonomous Driving
  8. 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving