mlsanityData to 14 Oct 2026

Cross-Stage Coherence in Hierarchical Driving VQA: Explicit Baselines and Learned Gated Context Projectors

arXiv 2604.22560 ↗First posted 24 Apr 2026cs.CV cs.AIcited by 0

On the map: End-to-end autonomous driving → VLMs for autonomous driving

Abstract (arXiv)

Graph Visual Question Answering (GVQA) for autonomous driving organizes reasoning into ordered stages, namely Perception, Prediction, and Planning, where planning decisions should remain consistent with the model's own perception. We present a comparative study of cross-stage context passing on DriveLM-nuScenes using two complementary mechanisms. The explicit variant evaluates three prompt-based conditioning strategies on a domain-adapted 4B VLM (Mini-InternVL2-4B-DA-DriveLM) without additional training, reducing NLI contradiction by up to 42.6% and establishing a strong zero-training baseline. The implicit variant introduces gated context projectors, which extract a hidden-state vector from one stage and inject a normalized, gated projection into the next stage's input embeddings. These projectors are jointly trained with stage-specific QLoRA adapters on a general-purpose 8B VLM (InternVL3-8B-Instruct) while updating only approximately 0.5% of parameters. The implicit variant achieves a statistically significant 34% reduction in planning-stage NLI contradiction (bootstrap 95% CIs, p < 0.05) and increases cross-stage entailment by 50%, evaluated with a multilingual NLI classifier to account for mixed-language outputs. Planning language quality also improves (CIDEr +30.3%), but lexical overlap and structural consistency degrade due to the absence of driving-domain pretraining. Since the two variants use different base models, we present them as complementary case studies: explicit context passing provides a strong training-free baseline for surface consistency, while implicit gated projection delivers significant planning-stage semantic gains, suggesting domain adaptation as a plausible next ingredient for full-spectrum improvement.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving. Builds on 32 of its 37 references that are in the map. First posted on arXiv 24 Apr 2026.

Builds on

32 of 37 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025VLAD: A VLM-Augmented Autonomous Driving Framework with Hierarchical Planning and Interpretable Decision Process2 Jul
  2. 2025Hierarchical Question-Answering for Driving Scene Understanding Using Vision-Language Models3 Jun
  3. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives7 Jan
  4. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
  5. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec

Other references · 27

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
  3. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  4. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  5. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  6. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  7. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 305
  8. 2025CODI: Compressing Chain-of-Thought into Continuous Space via Self-Distillationcited by 225
  9. 2025OpenDriveVLA: Towards End-to-end Autonomous Driving with Large Vision Language Action Modelcited by 176
  10. 2025Cosmos-Reason1: From Physical Common Sense To Embodied Reasoningcited by 164
  11. 2025SimLingo: Vision-Only Closed-Loop Autonomous Driving with Language-Action Alignmentcited by 150
  12. 2023DriveAdapter: Breaking the Coupling Barrier of Perception and Planning in End-to-End Autonomous Drivingcited by 144
15 more
  1. 2025DriveTransformer: Unified Transformer for Scalable End-to-End Autonomous Drivingcited by 126
  2. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  3. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performancecited by 94
  4. 2024OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Drivingcited by 93
  5. 2024DriveCoT: Integrating Chain-of-Thought Reasoning with End-to-End Drivingcited by 67
  6. 2025Distilling Multi-modal Large Language Models for Autonomous Drivingcited by 46
  7. 2025DriveLMM-o1: A Step-by-Step Reasoning Dataset and Large Multimodal Model for Driving Scenario Understandingcited by 36
  8. 2025HiP-AD: Hierarchical and Multi-Granularity Planning with Deformable Attention for Autonomous Driving in a Single Decodercited by 35
  9. 2024Hint-AD: Holistically Aligned Interpretability in End-to-End Autonomous Drivingcited by 33
  10. 2023Building Efficient Universal Classifiers with Natural Language Inferencecited by 22
  11. 2025VERDI: VLM-Embedded Reasoning for Autonomous Drivingcited by 16
  12. 2024Multi-Modal Adapter for Vision-Language Modelscited by 8
  13. 2025ALN-P3: Unified Language Alignment for Perception, Prediction, and Planning in Autonomous Drivingcited by 6
  14. 2025CogAD: Cognitive-Hierarchy Guided End-to-End Autonomous Drivingcited by 4
  15. 2024Precise Drive with VLM: First Prize Solution for PRCV 2024 Drive LM challengecited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025More Than Meets the Eye? Uncovering the Reasoning-Planning Disconnect in Training Vision-Language Driving Models
  2. 2026KLDrive: Fine-Grained 3D Scene Reasoning for Autonomous Driving based on Knowledge Graph
  3. 2025ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
  4. 2025Fine-Grained Evaluation of Large Vision-Language Models in Autonomous Driving
  5. 2026CausalDriveBench: Evaluating Causal Reasoning in Vision-Language-Action Models for Autonomous Driving
  6. 2025NuScenes-SpatialQA: A Spatial Understanding and Reasoning Benchmark for Vision-Language Models in Autonomous Driving
  7. 2025DriveRX: A Vision-Language Reasoning Model for Cross-Task Autonomous Driving
  8. 2026DRIVESPATIAL: A Benchmark for Spatiotemporal Intelligence in VLMs for Autonomous Driving