Towards Statistical Factuality Guarantee for Large Vision-Language Models
On the map: Multimodal LLMs / vision-language models → Hallucination mitigation in LVLMs
Abstract (arXiv)
Advancements in Large Vision-Language Models (LVLMs) have demonstrated promising performance in a variety of vision-language tasks involving image-conditioned free-form text generation. However, growing concerns about hallucinations in LVLMs, where the generated text is inconsistent with the visual context, are becoming a major impediment to deploying these models in applications that demand guaranteed reliability. In this paper, we introduce a framework to address this challenge, ConfLVLM, which is grounded on conformal prediction to achieve finite-sample distribution-free statistical guarantees on the factuality of LVLM output. This framework treats an LVLM as a hypothesis generator, where each generated text detail (or claim) is considered an individual hypothesis. It then applies a statistical hypothesis testing procedure to verify each claim using efficient heuristic uncertainty measures to filter out unreliable claims before returning any responses to users. We conduct extensive experiments covering three representative application domains, including general scene understanding, medical radiology report generation, and document understanding. Remarkably, ConfLVLM reduces the error rate of claims generated by LLaVa-1.5 for scene descriptions from 87.8% to 10.0% by filtering out erroneous claims with a 95.3% true positive rate. Our results further demonstrate that ConfLVLM is highly flexible, and can be applied to any black-box LVLMs paired with any uncertainty measure for any image-conditioned free-form text generation task while providing a rigorous guarantee on controlling the risk of hallucination.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 35 of its 53 references that are in the map. First posted on arXiv 27 Feb 2025.
Builds on
35 of 53 references.
Closest ancestors · 14
Same direction, newest first
- 2024Paying More Attention to Image: A Training-Free Method for Alleviating Hallucination in LVLMs31 Jul
- 2024THRONE: An Object-based Hallucination Benchmark for the Free-form Generations of Large Vision-Language Models8 May
- 2024Mitigating Hallucinations in Large Vision-Language Models with Instruction Contrastive Decoding27 Mar
- 2024Multi-Modal Hallucination Control by Visual Information Grounding20 Mar
- 2023Hallucination Augmented Contrastive Learning for Multimodal Large Language Model12 Dec
- 2023Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding28 Nov
- 2023Beyond Hallucinations: Enhancing LVLMs through Hallucination-Aware Direct Preference Optimization28 Nov
- 2023Volcano: Mitigating Multimodal Hallucination through Self-Feedback Guided Revision13 Nov
- 2023Woodpecker: Hallucination Correction for Multimodal Large Language Models24 Oct
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Models23 Oct
- 2023Negative Object Presence Evaluation (NOPE) to Measure Object Hallucination in Vision-Language Models9 Oct
- 2023Analyzing and Mitigating Object Hallucination in Large Vision-Language Models1 Oct
2 more
Other references · 21
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2023Visual Instruction Tuningcited by 7,869
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,917
- 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
- 2023A Survey on Multimodal Large Language Modelscited by 909
- 2023Aligning Large Multimodal Models with Factually Augmented RLHFcited by 712
- 2023CogAgent: A Visual Language Model for GUI Agentscited by 704
- 2018Object Hallucination in Image Captioningcited by 669
- 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 597
- 2023BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairscited by 520
- 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
9 more
- 2022Conformal prediction beyond exchangeabilitycited by 355
- 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
- 2023Conformal Prediction With Conditional Guaranteescited by 152
- 2024MAIRA-2: Grounded Radiology Report Generationcited by 145
- 2024Language Models with Conformal Factuality Guaranteescited by 131
- 2022Improving Chest X-Ray Report Generation by Leveraging Warm Startingcited by 95
- 2024Large language model validity via enhanced conformal prediction methodscited by 92
- 2023On the Road with GPT-4V(ision): Early Explorations of Visual-Language Model on Autonomous Drivingcited by 80
- 2024Do You Know What You Are Talking About? Characterizing Query-Knowledge Relevance For Reliable Retrieval Augmented Generationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026IntroConformal: Conformal Factuality Guarantees for Large Vision-Language Models via Introspective Signals
- 2026Reliability Challenges in Diffusion Vision-Language Models
- 2024VALOR-EVAL: Holistic Coverage and Faithfulness Evaluation of Large Vision-Language Models
- 2026Benchmarking Deflection and Hallucination in Large Vision-Language Models
- 2025Extracting Visual Facts from Intermediate Layers for Mitigating Hallucinations in Multimodal Large Language Models
- 2026Mitigating Visual Hallucinations in Multimodal Systems through Retrieval-Augmented Reliability-Aware Inference
- 2026V-DyKnow: A Dynamic Benchmark for Time-Sensitive Knowledge in Vision Language Models
- 2026CDH-Bench: A Commonsense-Driven Hallucination Benchmark for Evaluating Visual Fidelity in Vision-Language Models