# Evaluating and Enhancing Trustworthiness of LLMs in Perception Tasks

arXiv: https://arxiv.org/abs/2408.01433 · HTML: https://mlsanity.com/paper/2408.01433/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 19 of 19)

- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [AMBER: An LLM-free Multi-dimensional Benchmark for MLLMs Hallucination Evaluation](https://mlsanity.com/paper/2311.07397/) (2023)
- [Dolphins: Multimodal Language Model for Driving](https://mlsanity.com/paper/2312.00438/) (2023)
- [Evaluation and Analysis of Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2308.15126/) (2023)
- [Unified Hallucination Detection for Multimodal Large Language Models](https://mlsanity.com/paper/2402.03190/) (2024)
- [Learning to Localize Objects Improves Spatial Reasoning in Visual-LLMs](https://mlsanity.com/paper/2404.07449/) (2024)
- [Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation](https://mlsanity.com/paper/2401.15449/) (2024)
- [Exploring the Sensitivity of LLMs' Decision-Making Capabilities: Insights from Prompt Variation and Hyperparameters](https://mlsanity.com/paper/2312.17476/) (2023)
- [VLPD: Context-Aware Pedestrian Detection via Vision-Language Semantic Self-Supervision](https://mlsanity.com/paper/2304.03135/) (2023)
- [Applications of Large Scale Foundation Models for Autonomous Driving](https://mlsanity.com/paper/2311.12144/) (2023)
- [ChatGPT as a tool for User Story Quality Evaluation: Trustworthy Out of the Box?](https://mlsanity.com/paper/2306.12132/) (2023)
- [Hybrid Reasoning Based on Large Language Models for Autonomous Car Driving](https://mlsanity.com/paper/2402.13602/) (2024)
- [AutoHall: Automated Factuality Hallucination Dataset Generation for Large Language Models](https://mlsanity.com/paper/2310.00259/) (2023)
- [CarExpert: Leveraging Large Language Models for In-Car Conversational Question Answering](https://mlsanity.com/paper/2310.09536/) (2023)
- [Unsupervised Multi-view Pedestrian Detection](https://mlsanity.com/paper/2305.12457/) (2023)

## Built on by (most cited first, 4 of 4)

- [Mapping User Trust in Vision Language Models: Research Landscape, Challenges, and Prospects](https://mlsanity.com/paper/2505.05318/) (2025)
- [Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Support](https://mlsanity.com/paper/2606.00991/) (2026)
- [Can Urban Blight Be Accessed with Vision-language Models: A Case Study in Detroit](https://mlsanity.com/paper/2608.01753/) (2026)
- [Towards Systematic Qualification of Vision-Language Models for Automotive Perception Systems](https://mlsanity.com/paper/2609.25945/) (2026)

## Abstract (arXiv)

Today's advanced driver assistance systems (ADAS), like adaptive cruise control or rear collision warning, are finding broader adoption across vehicle classes. Integrating such advanced, multimodal Large Language Models (LLMs) on board a vehicle, which are capable of processing text, images, audio, and other data types, may have the potential to greatly enhance passenger comfort. Yet, an LLM's hallucinations are still a major challenge to be addressed. In this paper, we systematically assessed potential hallucination detection strategies for such LLMs in the context of object detection in vision-based data on the example of pedestrian detection and localization. We evaluate three hallucination detection strategies applied to two state-of-the-art LLMs, the proprietary GPT-4V and the open LLaVA, on two datasets (Waymo/US and PREPER CITY/Sweden). Our results show that these LLMs can describe a traffic situation to an impressive level of detail but are still challenged for further analysis activities such as object localization. We evaluate and extend hallucination detection approaches when applying these LLMs to video sequences in the example of pedestrian detection. Our experiments show that, at the moment, the state-of-the-art proprietary LLM performs much better than the open LLM. Furthermore, consistency enhancement techniques based on voting, such as the Best-of-Three (BO3) method, do not effectively reduce hallucinations in LLMs that tend to exhibit high false negatives in detecting pedestrians. However, extending the hallucination detection by including information from the past helps to improve results.