mlsanityData to 14 Oct 2026

Medical Hallucinations in Foundation Models and Their Impact on Healthcare

arXiv 2503.05777 ↗First posted 26 Feb 2025cs.CL cs.AI cs.CYcited by 93

On the map: LLM reasoning → Hallucination detection and mitigation

Abstract (arXiv)

Hallucinations in foundation models arise from autoregressive training objectives that prioritize token-likelihood optimization over epistemic accuracy, fostering overconfidence and poorly calibrated uncertainty. We define medical hallucination as any model-generated output that is factually incorrect, logically inconsistent, or unsupported by authoritative clinical evidence in ways that could alter clinical decisions. We evaluated 11 foundation models (7 general-purpose, 4 medical-specialized) across seven medical hallucination tasks spanning medical reasoning and biomedical information retrieval. General-purpose models achieved significantly higher proportions of hallucination-free responses than medical-specialized models (median: 76.6% vs 51.3%, difference = 25.2%, 95% CI: 18.7-31.3%, Mann-Whitney U = 27.0, p = 0.012, rank-biserial r = -0.64). Top-performing models such as Gemini-2.5 Pro exceeded 97% accuracy when augmented with chain-of-thought prompting (base: 87.6%), while medical-specialized models like MedGemma ranged from 28.6-61.9% despite explicit training on medical corpora. Chain-of-thought reasoning significantly reduced hallucinations in 86.4% of tested comparisons after FDR correction (q < 0.05), demonstrating that explicit reasoning traces enable self-verification and error detection. Physician audits confirmed that 64-72% of residual hallucinations stemmed from causal or temporal reasoning failures rather than knowledge gaps. A global survey of clinicians (n = 70) validated real-world impact: 91.8% had encountered medical hallucinations, and 84.7% considered them capable of causing patient harm. The underperformance of medical-specialized models despite domain training indicates that safety emerges from sophisticated reasoning capabilities and broad knowledge integration developed during large-scale pre-training, not from narrow optimization.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Placed in Hallucination detection and mitigation, one of 17 directions in LLM reasoning. First posted on arXiv 26 Feb 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 37; 2026: 56.

93 papers.

Sort:

By area:

  1. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  2. 2025MedBrowseComp: Benchmarking Medical Deep Research and Computer Usecited by 30
  3. 2025A Survey on Autonomy-Induced Security Risks in Large Model-Based Agentscited by 29
  4. 2026Baichuan-M3: Modeling Clinical Inquiry for Reliable Medical Decision-Makingcited by 11
  5. 2025Learned Hallucination Detection in Black-Box LLMs using Token-level Entropy Production Ratecited by 10
  6. 2026Knowledge Graphs are Implicit Reward Models: Path-Derived Signals Enable Compositional Reasoningcited by 7
  7. 2025Rethinking Retrieval-Augmented Generation for Medicine: A Large-Scale, Systematic Expert Evaluation and Practical Insightscited by 6
  8. 2025A Survey of Foundation Models for IoT: Taxonomy and Criteria-Based Analysiscited by 5
  9. 2025Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implicationscited by 5
  10. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucinationcited by 5
  11. 2026Self-Improvement of Large Language Models: A Technical Overview and Future Outlookcited by 5
  12. 2025Stabilizing Reasoning in Medical LLMs with Continued Pretraining and Reasoning Preference Optimizationcited by 4
38 more
  1. 2025Brittleness and Promise: Knowledge Graph Based Reward Modeling for Diagnostic Reasoningcited by 4
  2. 2025CCD: Mitigating Hallucinations in Radiology MLLMs via Clinical Contrastive Decodingcited by 4
  3. 2025Generalization of Fine-Tuned Uncertainty Communication and Metacognition in Large Language Modelscited by 4
  4. 2025BioMedSearch: A Multi-Source Biomedical Retrieval Framework Based on LLMscited by 4
  5. 2025Black-Box Behavioral Distillation Breaks Safety Alignment in Medical LLMscited by 4
  6. 2025MedCEG: Reinforcing Verifiable Medical Reasoning with Critical Evidence Graphcited by 4
  7. 2025Evaluating Clinical Competencies of Large Language Models with a General Practice Benchmarkcited by 3
  8. 2025GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoningcited by 3
  9. 2025Fine-Tuned LLMs Know They Don't Know: A Parameter-Efficient Approach to Recovering Honestycited by 3
  10. 2025WER is Unaware: Assessing How ASR Errors Distort Clinical Understanding in Patient Facing Dialoguecited by 3
  11. 2025Copy-Paste to Mitigate Large Language Model Hallucinationscited by 2
  12. 2025DispatchMAS: Fusing taxonomy and artificial intelligence agents for emergency medical servicescited by 2
  13. 2025Mitigating Hallucination in Large Language Models (LLMs): An Application-Oriented Survey on RAG, Reasoning, and Agentic Systemscited by 2
  14. 2025Foundation Models in Biomedical Imaging: Turning Hype into Realitycited by 2
  15. 2026Bayesian-LoRA: Probabilistic Low-Rank Adaptation of Large Language Modelscited by 2
  16. 2026CARE-RFT: Confidence-Anchored Reinforcement Finetuning for Reliable Reasoning in Large Language Modelscited by 2
  17. 2026RAG-X: Systematic Diagnosis of Retrieval-Augmented Generation for Medical Question Answeringcited by 2
  18. 2026Computational Pathology in the Era of Emerging Foundation and Agentic AI -- International Expert Perspectives on Clinical Integration and Translational Readinesscited by 2
  19. 2026Dialectic-Med: Mitigating Diagnostic Hallucinations via Counterfactual Adversarial Multi-Agent Debatecited by 2
  20. 2026Med-StepBench: A Hierarchical Reasoning Framework for Evaluating Hallucinations in Medical Vision-Language Modelscited by 2
  21. 2026ClinHallu: A Benchmark for Diagnosing Stage-Wise Hallucinations in Medical MLLM Reasoningcited by 2
  22. 2026The Path to Self-Evolving Clinical Systems: Scaling Medical Agents from Assistance to Autonomycited by 2
  23. 2025ReLoop: "Seeing Twice and Thinking Backwards" via Closed-loop Training to Mitigate Hallucinations in Multimodal understandingcited by 1
  24. 2025SynthEHR-Eviction: Enhancing Eviction SDoH Detection with LLM-Augmented Synthetic EHR Datacited by 1
  25. 2025Zoom In Disparities in Healthcare LLM Q&Acited by 1
  26. 2025Demo: Statistically Significant Results On Biases and Errors of LLMs Do Not Guarantee Generalizable Resultscited by 1
  27. 2025Mitigating hallucinations in healthcare LLMs with granular fact-checking and domain-specific adaptationcited by 1
  28. 2026AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domainscited by 1
  29. 2026PanCanBench: A Comprehensive Benchmark for Evaluating Large Language Models in Pancreatic Oncologycited by 1
  30. 2026An Alternative Trajectory for Generative AIcited by 1
  31. 2026Process Reward Agents for Steering Knowledge-Intensive Reasoningcited by 1
  32. 2026An AI Co-Data-Scientist for Prioritizing Candidate Biomarkers from Wearable Sensor Datacited by 1
  33. 2026FeatEHR-LLM: Leveraging Large Language Models for Feature Engineering in Electronic Health Recordscited by 1
  34. 2026BELIEF: Structured Evidence Modeling and Uncertainty-Aware Fusion for Biomedical Question Answeringcited by 1
  35. 2026A Wolf in Sheep's Clothing: Targeted Routing Hijacking in Federated RAGcited by 1
  36. 2026Med-HEAL: Analyzing and Mitigating Hallucinations in Medical LLMs with Hallucination-Aware In-Context Learningcited by 1
  37. 2026Baichuan-M4: A Clinical-Grade Medical Agent System for Continuous Carecited by 1
  38. 2026Aligning Clinical Needs and AI Capabilities: A Survey on LLMs for Medical Reasoningcited by 1

50 most cited of 93 All 93 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MedHallBench: A New Benchmark for Assessing Hallucination in Medical Large Language Models
  2. 2025Diagnosing Hallucination Risk in AI Surgical Decision-Support: A Sequential Framework for Sequential Validation
  3. 2024MedVH: Towards Systematic Evaluation of Hallucination for Large Vision Language Models in the Medical Context
  4. 2025MedHEval: Benchmarking Hallucinations and Mitigation Strategies in Medical Large Vision-Language Models
  5. 2025MedHallTune: An Instruction-Tuning Benchmark for Mitigating Medical Hallucination in Vision-Language Models
  6. 2024Detecting and Evaluating Medical Hallucinations in Large Vision Language Models
  7. 2026Deterministic Hallucination Detection in Medical VQA via Confidence-Evidence Bayesian Gain
  8. 2026Hallucination in Medical Imaging AI: A Cross-Modality Analytical Framework for Taxonomy, Detection, and Mitigation under Regulatory Constraints