# Adversarially-Aware Architecture Design for Robust Medical AI Systems

arXiv: https://arxiv.org/abs/2510.23622 · HTML: https://mlsanity.com/paper/2510.23622/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 11 references that are in the map.
- First posted on arXiv 23 Oct 2025.

## Builds on (most cited first, 7 of 7)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)

## Abstract (arXiv)

Adversarial attacks pose a severe risk to AI systems used in healthcare, capable of misleading models into dangerous misclassifications that can delay treatments or cause misdiagnoses. These attacks, often imperceptible to human perception, threaten patient safety, particularly in underserved populations. Our study explores these vulnerabilities through empirical experimentation on a dermatological dataset, where adversarial methods significantly reduce classification accuracy. Through detailed threat modeling, experimental benchmarking, and model evaluation, we demonstrate both the severity of the threat and the partial success of defenses like adversarial training and distillation. Our results show that while defenses reduce attack success rates, they must be balanced against model performance on clean data. We conclude with a call for integrated technical, ethical, and policy-based approaches to build more resilient, equitable AI in healthcare.