mlsanityData to 7 Oct 2026

Medical Foundation Models are Susceptible to Targeted Misinformation Attacks

arXiv 2309.17007 ↗First posted 29 Sep 2023cs.LG cs.AI cs.CYcited by 5

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have broad medical knowledge and can reason about medical information across many domains, holding promising potential for diverse medical applications in the near future. In this study, we demonstrate a concerning vulnerability of LLMs in medicine. Through targeted manipulation of just 1.1% of the model's weights, we can deliberately inject an incorrect biomedical fact. The erroneous information is then propagated in the model's output, whilst its performance on other biomedical tasks remains intact. We validate our findings in a set of 1,038 incorrect biomedical facts. This peculiar susceptibility raises serious security and trustworthiness concerns for the application of LLMs in healthcare settings. It accentuates the need for robust protective measures, thorough verification mechanisms, and stringent management of access to these models, ensuring their reliable and safe use in medical practice.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 22 of its 46 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

22 of 46 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
  11. 2022Large Language Models Encode Clinical Knowledgecited by 1,807
  12. 2022Mass-Editing Memory in a Transformercited by 976
10 more
  1. 2022Using cognitive psychology to understand GPT-3cited by 389
  2. 2023A Comprehensive Survey on Pretrained Foundation Models: A History from BERT to ChatGPTcited by 364
  3. 2023MedAlpaca -- An Open-Source Collection of Medical Conversational AI Models and Training Datacited by 282
  4. 2021Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Docited by 178
  5. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  6. 2022The Debate Over Understanding in AI's Large Language Modelscited by 142
  7. 2021Towards artificial general intelligence via a multimodal foundation modelcited by 112
  8. 2022Toward Next-Generation Artificial Intelligence: Catalyzing the NeuroAI Revolutioncited by 78
  9. 2022TalkToModel: Explaining Machine Learning Models with Interactive Natural Language Conversationscited by 65
  10. 2020Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalizationcited by 12

Built on by

Cited by year

Citing papers per year: 2023: 1; 2024: 4.

5 papers.

  1. 2023Large Models for Time Series and Spatio-Temporal Data: A Survey and Outlookcited by 109
  2. 2024What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detectioncited by 24
  3. 2024A Survey on Trustworthiness in Foundation Models for Medical Image Analysiscited by 16
  4. 2024PromptSmooth: Certifying Robustness of Medical Vision-Language Models via Prompt Learningcited by 16
  5. 2024Beyond Multiple-Choice Accuracy: Real-World Challenges of Implementing Large Language Models in Healthcarecited by 4