# RadFlag: A Black-Box Hallucination Detection Method for Medical Vision Language Models

arXiv: https://arxiv.org/abs/2411.00299 · HTML: https://mlsanity.com/paper/2411.00299/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Hallucination mitigation in LVLMs, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 1 Nov 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Hallucination mitigation in LVLMs](https://mlsanity.com/mllm/hallucination-mitigation-in-lvlms/)

## Builds on (most cited first, 13 of 13)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [Hallucination of Multimodal Large Language Models: A Survey](https://mlsanity.com/paper/2404.18930/) (2024)
- [A Survey on Hallucination in Large Vision-Language Models](https://mlsanity.com/paper/2402.00253/) (2024)
- [RadGraph: Extracting Clinical Entities and Relations from Radiology Reports](https://mlsanity.com/paper/2106.14463/) (2021)
- [Interactive and Explainable Region-guided Radiology Report Generation](https://mlsanity.com/paper/2304.08295/) (2023)
- [Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigation](https://mlsanity.com/paper/2305.15852/) (2023)
- [Uncertainty Estimation and Quantification for LLMs: A Simple Supervised Approach](https://mlsanity.com/paper/2404.15993/) (2024)
- [Improving Radiology Report Generation Systems by Removing Hallucinated References to Non-existent Priors](https://mlsanity.com/paper/2210.06340/) (2022)
- [MedVersa: A Generalist Foundation Model for Medical Image Interpretation](https://mlsanity.com/paper/2405.07988/) (2024)
- [Quantifying and Leveraging Predictive Uncertainty for Medical Image Assessment](https://mlsanity.com/paper/2007.04258/) (2020)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)

## Built on by (most cited first, 16 of 16)

- [Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities](https://mlsanity.com/paper/2508.07031/) (2025)
- [Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation](https://mlsanity.com/paper/2604.02543/) (2026)
- [UniVRSE: Unified Vision-conditioned Response Semantic Entropy for Hallucination Detection in Medical Vision-Language Models](https://mlsanity.com/paper/2503.20504/) (2025)
- [Phrase-grounded Fact-checking for Automatically Generated Chest X-ray Reports](https://mlsanity.com/paper/2509.21356/) (2025)
- [Hallucination Filtering in Radiology Vision-Language Models Using Discrete Semantic Entropy](https://mlsanity.com/paper/2510.09256/) (2025)
- [V-Loop: Visual Logical Loop Verification for Hallucination Detection in Medical Visual Question Answering](https://mlsanity.com/paper/2601.18240/) (2026)
- [Process Reward Models for Sentence-Level Verification of LVLM Radiology Reports](https://mlsanity.com/paper/2510.23217/) (2025)
- [From Attribution to Abstention: Training-Free Attention-Based Auditing for Clinical Summarization](https://mlsanity.com/paper/2601.16397/) (2026)
- [Can Agents Distinguish Visually Hard-to-Separate Diseases in a Zero-Shot Setting? A Pilot Study](https://mlsanity.com/paper/2602.22959/) (2026)
- [CuraView: A Multi-Agent Framework for Medical Hallucination Detection with GraphRAG-Enhanced Knowledge Verification](https://mlsanity.com/paper/2605.03476/) (2026)
- [VIHD: Visual Intervention-based Hallucination Detection for Medical Visual Question Answering](https://mlsanity.com/paper/2605.20772/) (2026)
- [Hallucination Detection and Correction in Medical VLMs via Counter-Evidence Verification](https://mlsanity.com/paper/2606.18609/) (2026)
- [A Benchmark for Hallucination Detection in VLMs for Gastrointestinal Endoscopy](https://mlsanity.com/paper/2606.24115/) (2026)
- [Visual Semantic Entropy: Do Vision Language Models Recognize Visual Ambiguity?](https://mlsanity.com/paper/2606.31407/) (2026)
- [Learning to See Locally and Align Clinically with Pathology Semantics for Radiology Report Generation](https://mlsanity.com/paper/2608.00279/) (2026)
- [Auditing Medical Vision-Language Models on Chest Radiographs: Estimating Reference Agreement Across Institutions](https://mlsanity.com/paper/2608.07550/) (2026)

## Abstract (arXiv)

Generating accurate radiology reports from medical images is a clinically important but challenging task. While current Vision Language Models (VLMs) show promise, they are prone to generating hallucinations, potentially compromising patient care. We introduce RadFlag, a black-box method to enhance the accuracy of radiology report generation. Our method uses a sampling-based flagging technique to find hallucinatory generations that should be removed. We first sample multiple reports at varying temperatures and then use a Large Language Model (LLM) to identify claims that are not consistently supported across samples, indicating that the model has low confidence in those claims. Using a calibrated threshold, we flag a fraction of these claims as likely hallucinations, which should undergo extra review or be automatically rejected. Our method achieves high precision when identifying both individual hallucinatory sentences and reports that contain hallucinations. As an easy-to-use, black-box system that only requires access to a model's temperature parameter, RadFlag is compatible with a wide range of radiology report generation models and has the potential to broadly improve the quality of automated radiology reporting.