# Towards Systematic Qualification of Vision-Language Models for Automotive Perception Systems

arXiv: https://arxiv.org/abs/2609.25945 · HTML: https://mlsanity.com/paper/2609.25945/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Driving VLMs and open-vocabulary detection, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 24 of its 41 references that are in the map.
- First posted on arXiv 22 Sep 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Driving VLMs and open-vocabulary detection](https://mlsanity.com/mllm/driving-vlms-and-open-vocabulary-detection/)

## Builds on (most cited first, 24 of 24)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Ignore Previous Prompt: Attack Techniques For Language Models](https://mlsanity.com/paper/2211.09527/) (2022)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [End-to-end Autonomous Driving: Challenges and Frontiers](https://mlsanity.com/paper/2306.16927/) (2023)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models](https://mlsanity.com/paper/1810.12715/) (2018)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective](https://mlsanity.com/paper/2302.12095/) (2023)
- [A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validation](https://mlsanity.com/paper/2305.11391/) (2023)
- [Feature-Guided Black-Box Safety Testing of Deep Neural Networks](https://mlsanity.com/paper/1710.07859/) (2017)
- [Testing Deep Neural Networks](https://mlsanity.com/paper/1803.04792/) (2018)
- [CrossCheckGPT: Universal Hallucination Ranking for Multimodal Foundation Models](https://mlsanity.com/paper/2405.13684/) (2024)
- [FactSelfCheck: Fact-Level Black-Box Hallucination Detection for LLMs](https://mlsanity.com/paper/2503.17229/) (2025)
- [Safety Analysis in the Era of Large Language Models: A Case Study of STPA using ChatGPT](https://mlsanity.com/paper/2304.01246/) (2023)
- [Hybrid Reasoning Based on Large Language Models for Autonomous Car Driving](https://mlsanity.com/paper/2402.13602/) (2024)
- [Systematic Literature Review of Validation Methods for AI Systems](https://mlsanity.com/paper/2107.12190/) (2021)
- [CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering](https://mlsanity.com/paper/2310.09536/) (2023)
- [Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks](https://mlsanity.com/paper/2408.01433/) (2024)
- [LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Tasks](https://mlsanity.com/paper/2409.12580/) (2024)

## Abstract (arXiv)

The field of Artificial Intelligence has been adopted for many application domains. Vision Language Models are one of the recently advanced AI techniques that have been explored to support automotive features such as vehicle perception, and safety assurance. However, such language models are prone to hallucinations, posing a potential threat to the safety of automotive systems that may incorporate them. Within the automotive domain, VLMs could not only hallucinate traffic objects, but could also fail to identify traffic objects that are actually present, which may potentially lead to dangerous situations. Though we have observed a growing body of literature that proposes verification and validation techniques for safe and trustworthy AI, these methods are often studied in isolation, focusing either on run-time or design-time phases. Such isolated techniques could be insufficient in safety-critical, realistic contexts such as automotive perception systems. In this paper, we analyze design-time and run-time verification and validation techniques based on a taxonomy presented by Huang et al. We present an automotive study in which a design-time qualification workflow is proposed to complement run-time monitoring. This workflow combines a fixed safety-relevant ontology-based structured annotation system together with a synonym-based evaluation process to statistically evaluate three state-of-the-art VLMs against data from the nuScenes dataset. We observed that the proposed technique enables deterministic and repeatable quantification of the hallucinations VLMs generate in automotive perception-related tasks. The proposed workflow supports model comparison and deployment-oriented engineering decisions within the design-time verification and validation process and will contribute to a holistic verification strategy that strives towards trustworthy automotive perception systems