mlsanityData to 14 Oct 2026

What to Test Next: Interpretable Coverage Gap Discovery in Driving VLMs

arXiv 2606.01624 ↗First posted 1 Jun 2026cs.CV cs.SEcited by 0

On the map: Multimodal LLMs / vision-language models → Driving VLMs and open-vocabulary detection

also in 1 more
Abstract (arXiv)

Driving vision-language models (VLMs) must accurately understand scenes across diverse conditions defined by Operational Design Domains (ODDs), yet verification remains sparse: many slices are missing, making empirical failure rates unreliable. We propose SliceScorer, a deterministic scoring rule for missing-slice recommendation that combines (i) an exposure-based coverage prior to prioritize rare, under-tested regions, and (ii) a neighbor-failure prior that propagates risk from similar tested conditions. SliceScorer is deliberately simple - interpretable, auditable, and conservative - properties essential for safety-critical validation. For stress testing beyond the declared ODD, we embed SliceScorer within SliceNav, an LLM-orchestrated verification pipeline where the model interprets developer queries to select relevant operators (triage, scoring, acquisition, evaluation) and vocabulary extensions, composing verification workflows while keeping all scoring deterministic and auditable. Experiments on three driving VLMs (WiseAD, DriveMM, Cosmos-Reason2-2B) show that SliceNav surfaces high-risk coverage gaps more effectively than prior slice-discovery methods while maintaining diverse recommendations across the condition space. Ablations confirm both scoring components contribute, and qualitative analysis demonstrates end-to-end workflows from developer query to targeted evaluation.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 27 of its 41 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

27 of 41 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Model13 Dec
  2. 2024ContextVLM: Zero-Shot and Few-Shot Context Understanding for Autonomous Driving using Vision Language Models30 Aug
  3. 2023LingoQA: Visual Question Answering for Autonomous Driving21 Dec
  4. 2023WEDGE: A multi-weather autonomous driving dataset built from generative vision-language models12 May

Other references · 23

  1. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,416
  5. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,288
  6. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 897
  8. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 599
  9. 2021NuPlan: A closed-loop ML-based planning benchmark for autonomous vehiclescited by 493
  10. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 455
  11. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 156
  12. 2022DRAMA: Joint Risk Localization and Captioning in Drivingcited by 126
11 more
  1. 2020RADIATE: A Radar Dataset for Automotive Perception in Bad Weathercited by 100
  2. 2020Canadian Adverse Driving Conditions Datasetcited by 87
  3. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  4. 2023Zenseact Open Dataset: A large-scale and diverse multimodal dataset for autonomous drivingcited by 83
  5. 2018Automated Data Slicing for Model Validation:A Big data - AI Integration Approachcited by 70
  6. 2020DAWN: Vehicle Detection in Adverse Weather Nature Datasetcited by 41
  7. 2025TUMTraffic-VideoQA: A Benchmark for Unified Spatio-Temporal Video Understanding in Traffic Scenescited by 23
  8. 2024ROADWork: A Dataset and Benchmark for Learning to Recognize, Observe, Analyze and Drive Through Work Zonescited by 22
  9. 2022PeSOTIF: a Challenging Visual Dataset for Perception SOTIF Problems in Long-tail Traffic Scenarioscited by 8
  10. 2024Coverage Metrics for a Scenario Database for the Scenario-Based Assessment of Automated Driving Systemscited by 4
  11. 2022AutoSlicer: Scalable Automated Data Slicing for ML Model Analysiscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Are VLMs Ready for Autonomous Driving? An Empirical Study from the Reliability, Data, and Metric Perspectives
  2. 2026Towards Reliable Vision-Language Models for Autonomous Driving
  3. 2026Revealing Interpretable Failure Modes of VLMs
  4. 2024AutoTrust: Benchmarking Trustworthiness in Large Vision Language Models for Autonomous Driving
  5. 2025Can VLMs Unlock Semantic Anomaly Detection? A Framework for Structured Reasoning
  6. 2025Structured Labeling Enables Faster Vision-Language Models for End-to-End Autonomous Driving
  7. 2026Answer-Level Trust Selection for Physical Vision-Language Reasoning
  8. 2025Attention, Please! PixelSHAP Reveals What Vision-Language Models Actually Focus On