# Trustworthy Medical Imaging with Large Language Models: A Study of Hallucinations Across Modalities

arXiv: https://arxiv.org/abs/2508.07031 · HTML: https://mlsanity.com/paper/2508.07031/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 32 references that are in the map.
- First posted on arXiv 9 Aug 2025.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation](https://mlsanity.com/paper/2305.11490/) (2023)
- [Creating Trustworthy LLMs: Dealing with Hallucinations in Healthcare AI](https://mlsanity.com/paper/2311.01463/) (2023)
- [Towards a Holistic Framework for Multimodal Large Language Models in Three-dimensional Brain CT Report Generation](https://mlsanity.com/paper/2407.02235/) (2024)
- [On Large Visual Language Models for Medical Imaging Analysis: An Empirical Study](https://mlsanity.com/paper/2402.14162/) (2024)
- [Hallucination Index: An Image Quality Metric for Generative Reconstruction Models](https://mlsanity.com/paper/2407.12780/) (2024)
- [MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context](https://mlsanity.com/paper/2407.02730/) (2024)
- [RadFlag: A Black-Box Hallucination Detection Method for Medical Vision Language Models](https://mlsanity.com/paper/2411.00299/) (2024)
- [MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models](https://mlsanity.com/paper/2412.18947/) (2024)
- [FactCheXcker: Mitigating Measurement Hallucinations in Chest X-ray Report Generation Models](https://mlsanity.com/paper/2411.18672/) (2024)
- [ReXTrust: A Model for Fine-Grained Hallucination Detection in AI-Generated Radiology Reports](https://mlsanity.com/paper/2412.15264/) (2024)
- [Tackling Hallucination from Conditional Models for Medical Image Reconstruction with DynamicDPS](https://mlsanity.com/paper/2503.01075/) (2025)
- [Reducing Hallucinations of Medical Multimodal Large Language Models with Visual Retrieval-Augmented Generation](https://mlsanity.com/paper/2502.15040/) (2025)
- [Prompt-Guided Generation of Structured Chest X-Ray Report Using a Pre-trained LLM](https://mlsanity.com/paper/2404.11209/) (2024)
- [Addressing Image Hallucination in Text-to-Image Generation through Factual Image Retrieval](https://mlsanity.com/paper/2407.10683/) (2024)
- [Hallucinations and Key Information Extraction in Medical Texts: A Comprehensive Assessment of Open-Source Large Language Models](https://mlsanity.com/paper/2504.19061/) (2025)
- [Can Large Language Models Challenge CNNs in Medical Image Analysis?](https://mlsanity.com/paper/2505.23503/) (2025)

## Built on by (most cited first, 7 of 7)

- [A co-evolving agentic AI system for medical imaging analysis](https://mlsanity.com/paper/2509.20279/) (2025)
- [SkinCLIP-VL: Consistency-Aware Vision-Language Learning for Multimodal Skin Cancer Diagnosis](https://mlsanity.com/paper/2603.21010/) (2026)
- [Glaucoma Detection and Structured OCT Report Generation via a Fine-tuned Multimodal Large Language Model](https://mlsanity.com/paper/2510.02403/) (2025)
- [NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders](https://mlsanity.com/paper/2603.24846/) (2026)
- [NEURO-GUARD: Neuro-Symbolic Generalization and Unbiased Adaptive Routing for Diagnostics -- Explainable Medical AI](https://mlsanity.com/paper/2512.18177/) (2025)
- [Adversarial Reframing: A Framework for Targeted Generation in Language Models](https://mlsanity.com/paper/2605.21674/) (2026)
- [From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learning](https://mlsanity.com/paper/2609.38924/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly applied to medical imaging tasks, including image interpretation and synthetic image generation. However, these models often produce hallucinations, which are confident but incorrect outputs that can mislead clinical decisions. This study examines hallucinations in two directions: image to text, where LLMs generate reports from X-ray, CT, or MRI scans, and text to image, where models create medical images from clinical prompts. We analyze errors such as factual inconsistencies and anatomical inaccuracies, evaluating outputs using expert informed criteria across imaging modalities. Our findings reveal common patterns of hallucination in both interpretive and generative tasks, with implications for clinical reliability. We also discuss factors contributing to these failures, including model architecture and training data. By systematically studying both image understanding and generation, this work provides insights into improving the safety and trustworthiness of LLM driven medical imaging systems.