mlsanityData to 7 Oct 2026

HMVLM: Multistage Reasoning-Enhanced Vision-Language Model for Long-Tailed Driving Scenarios

arXiv 2506.05883 ↗First posted 6 Jun 2025cs.CV cs.AIcited by 14

On the map: LLM reasoning → Chain-of-thought reasoning for driving

also in 2 more
Abstract (arXiv)

We present HaoMo Vision-Language Model (HMVLM), an end-to-end driving framework that implements the slow branch of a cognitively inspired fast-slow architecture. A fast controller outputs low-level steering, throttle, and brake commands, while a slow planner-a large vision-language model-generates high-level intents such as "yield to pedestrian" or "merge after the truck" without compromising latency. HMVLM introduces three upgrades: (1) selective five-view prompting with an embedded 4s history of ego kinematics, (2) multi-stage chain-of-thought (CoT) prompting that enforces a Scene Understanding -> Driving Decision -> Trajectory Inference reasoning flow, and (3) spline-based trajectory post-processing that removes late-stage jitter and sharp turns. Trained on the Waymo Open Dataset, these upgrades enable HMVLM to achieve a Rater Feedback Score (RFS) of 7.7367, securing 2nd place in the 2025 Waymo Vision-based End-to-End (E2E) Driving Challenge and surpassing the public baseline by 2.77%.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Chain-of-thought reasoning for driving, one of 17 directions in LLM reasoning. Builds on 11 of its 14 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

11 of 14 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning10 Mar

Other references · 10

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  3. 2025Qwen2.5-VL Technical Reportcited by 5,418
  4. 2023VAD: Vectorized Scene Representation for Efficient Autonomous Drivingcited by 613
  5. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Modelscited by 482
  6. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 446
  7. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  8. 2024A Survey on Hallucination in Large Vision-Language Modelscited by 352
  9. 2024EMMA: End-to-End Multimodal Model for Autonomous Drivingcited by 226
  10. 2024Senna: Bridging Large Vision-Language Models and End-to-End Autonomous Drivingcited by 156

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 11.

14 papers.

Sort:

By area:

  1. 2025WOD-E2E: Waymo Open Dataset for End-to-End Driving in Challenging Long-tail Scenarioscited by 60
  2. 2025Vision-Language-Action Models for Autonomous Driving: Past, Present, and Futurecited by 32
  3. 2026NoRD: A Data-Efficient Vision-Language-Action Model that Drives without Reasoningcited by 15
  4. 2025Poutine: Vision-Language-Trajectory Pre-Training and Reinforcement Learning Post-Training Enable Robust End-to-End Autonomous Drivingcited by 11
  5. 2026Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Drivingcited by 8
  6. 2026MindVLA-U1: VLA Beats VA with Unified Streaming Architecture for Autonomous Drivingcited by 6
  7. 2026SimWAM: A Simple World Action Model for End-to-End Autonomous Drivingcited by 5
  8. 2026DriveMA: Driving Vision-Language-Action Models with verifiable Meta-Actionscited by 2
  9. 2026DriveMA: Rethinking Language Interfaces in Driving VLAs with One-Step Meta-Actionscited by 1
  10. 2026NTR: Neural Token Reconstruction for Scene Token Bottleneck in End-to-End Drivingcited by 1
  11. 2026Action Emergence from Streaming Intentcited by 0
  12. 2026SUV: Future Scene Understanding as Video Generation for End-to-End Drivingcited by 0
2 more
  1. 2026CoLT-Drive: Counterfactual Long-Tail Benchmarking and Knowledge-Preserving Adaptation for Driving Affordance Predictioncited by 0
  2. 2026Beyond Textual Chain-of-Thought: A Survey on Action-Grounded Reasoning in Autonomous Drivingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025LLaViDA: A Large Language Vision Driving Assistant for Explicit Reasoning and Enhanced Trajectory Planning
  2. 2025VLMPlanner: Integrating Visual Language Models with Motion Planning
  3. 2026SteerVLA: Steering Vision-Language-Action Models in Long-Tail Driving Scenarios
  4. 2025LMAD: Integrated End-to-End Vision-Language Model for Explainable Autonomous Driving
  5. 2025ColaVLA: Leveraging Cognitive Latent Reasoning for Hierarchical Parallel Trajectory Planning in Autonomous Driving
  6. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
  7. 2026NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving
  8. 2025Drive-R1: Bridging Reasoning and Planning in VLMs for Autonomous Driving with Reinforcement Learning