mlsanityData to 7 Oct 2026

Nav-EE: Navigation-Guided Early Exiting for Efficient Vision-Language Models in Autonomous Driving

arXiv 2510.01795 ↗First posted 2 Oct 2025cs.RO cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 2 more
Abstract (arXiv)

Vision-Language Models (VLMs) are increasingly applied in autonomous driving for unified perception and reasoning, but high inference latency hinders real-time deployment. Early-exit reduces latency by terminating inference at intermediate layers, yet its task-dependent nature limits generalization across diverse scenarios. We observe that this limitation aligns with autonomous driving: navigation systems can anticipate upcoming contexts (e.g., intersections, traffic lights), indicating which tasks will be required. We propose Nav-EE, a navigation-guided early-exit framework that precomputes task-specific exit layers offline and dynamically applies them online based on navigation priors. Experiments on CODA, Waymo, and BOSCH show that Nav-EE achieves accuracy comparable to full inference while reducing latency by up to 63.9%. Real-vehicle integration with Autoware Universe further demonstrates reduced inference latency (600ms to 300ms), supporting faster decision-making in complex scenarios. These results suggest that coupling navigation foresight with early-exit offers a viable path toward efficient deployment of large models in autonomous systems. Code and data are available at our anonymous repository: https://anonymous.4open.science/r/Nav-EE-BBC4

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 30 of its 41 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

30 of 41 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025DynRsl-VLM: Enhancing Autonomous Driving Perception with Dynamic Resolution Vision-Language Models14 Mar
  2. 2024DriveVLM: The Convergence of Autonomous Driving and Large Vision-Language Models19 Feb
  3. 2023DriveLM: Driving with Graph Visual Question Answering21 Dec
  4. 2023Dolphins: Multimodal Language Model for Driving1 Dec
  5. 2023Talk2BEV: Language-enhanced Bird's-eye View Maps for Autonomous Driving3 Oct

Other references · 25

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  4. 2019Scalability in Perception for Autonomous Driving: Waymo Open Datasetcited by 2,436
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  7. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 564
  8. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 559
  9. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Modelscited by 456
  10. 2023LMDrive: Closed-Loop End-to-End Driving with Large Language Modelscited by 301
  11. 2022Confident Adaptive Language Modelingcited by 297
  12. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
13 more
  1. 2020FastBERT: a Self-distilling BERT with Adaptive Inference Timecited by 224
  2. 2021Causal Inference in Natural Language Processing: Estimation, Prediction, Interpretation and Beyondcited by 177
  3. 2022CODA: A Real-World Road Corner Case Dataset for Object Detection in Autonomous Drivingcited by 100
  4. 2023Fast and Robust Early-Exiting Framework for Autoregressive Language Models with Synchronized Parallel Decodingcited by 80
  5. 2024FastVLM: Efficient Vision Encoding for Vision Language Modelscited by 76
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 36
  7. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
  8. 2021Accelerating BERT Inference for Sequence Labeling via Early-Exitcited by 29
  9. 2023Adaptive Gating in Mixture-of-Experts based Language Modelscited by 16
  10. 2024Text-to-Drive: Diverse Driving Behavior Synthesis via Large Language Modelscited by 8
  11. 2022Understanding the Robustness of Multi-Exit Models under Common Corruptionscited by 4
  12. 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environmentscited by 4
  13. 2025PoTPTQ: A Two-step Power-of-Two Post-training for LLMscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025AD-EE: Early Exiting for Fast and Reliable Vision-Language Models in Autonomous Driving
  2. 2025NavigScene: Bridging Local Perception and Global Navigation for Beyond-Visual-Range Autonomous Driving
  3. 2025Goal-Based Vision-Language Driving
  4. 2026NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving
  5. 2025AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning
  6. 2026Rethinking Language's Role in Efficient VLA for Autonomous Vehicles: Toward Smarter, Trustworthy Driving
  7. 2025ViLaD: A Large Vision Language Diffusion Framework for End-to-End Autonomous Driving
  8. 2026Latent-Centroid Steering: Single-Pass Classifier-Free Guidance for Command-Aligned Autonomous Driving