# A Systematic Analysis of Declining Medical Safety Messaging in Generative AI Models

arXiv: https://arxiv.org/abs/2507.08030 · HTML: https://mlsanity.com/paper/2507.08030/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 4 of its 19 references that are in the map.
- First posted on arXiv 8 Jul 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 4 of 4)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [The Need for Guardrails with Large Language Models in Medical Safety-Critical Settings: An Artificial Intelligence Application in the Pharmacovigilance Ecosystem](https://mlsanity.com/paper/2407.18322/) (2024)
- [Medical Misinformation in AI-Assisted Self-Diagnosis: Development of a Method (EvalPrompt) for Analyzing Large Language Models](https://mlsanity.com/paper/2307.04910/) (2023)
- [People over trust AI-generated medical responses and view them to be as valid as doctors, despite low accuracy](https://mlsanity.com/paper/2408.15266/) (2024)

## Built on by (most cited first, 1 of 1)

- [AI Identity Disclosure Under Professional Personas: A Gap Between Capacity and Consistency](https://mlsanity.com/paper/2511.21569/) (2025)

## Abstract (arXiv)

Generative AI models, including large language models (LLMs) and vision-language models (VLMs), are increasingly used to interpret medical images and answer clinical questions. Their responses often include inaccuracies; therefore, safety measures like medical disclaimers are critical to remind users that AI outputs are not professionally vetted or a substitute for medical advice. This study evaluated the presence of disclaimers in LLM and VLM outputs across model generations from 2022 to 2025. Using 500 mammograms, 500 chest X-rays, 500 dermatology images, and 500 medical questions, outputs were screened for disclaimer phrases. Medical disclaimer presence in LLM and VLM outputs dropped from 26.3% in 2022 to 0.97% in 2025, and from 19.6% in 2023 to 1.05% in 2025, respectively. By 2025, the majority of models displayed no disclaimers. As public models become more capable and authoritative, disclaimers must be implemented as a safeguard adapting to the clinical context of each output.