# Systems-Theoretic and Data-Driven Security Analysis in ML-enabled Medical Devices

arXiv: https://arxiv.org/abs/2506.15028 · HTML: https://mlsanity.com/paper/2506.15028/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 117 references that are in the map.
- First posted on arXiv 18 Jun 2025.

## Builds on (most cited first, 23 of 23)

- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Tactics of Adversarial Attack on Deep Reinforcement Learning Agents](https://mlsanity.com/paper/1703.06748/) (2017)
- [Understanding Adversarial Attacks on Deep Learning Based Medical Image Analysis Systems](https://mlsanity.com/paper/1907.10456/) (2019)
- [Adversarial Attacks Against Medical Deep Learning Systems](https://mlsanity.com/paper/1804.05296/) (2018)
- [TELeR: A General Taxonomy of LLM Prompts for Benchmarking Complex Tasks](https://mlsanity.com/paper/2305.11430/) (2023)
- [Adversarial Attack Vulnerability of Medical Image Analysis Systems: Unexplored Factors](https://mlsanity.com/paper/2006.06356/) (2020)
- [Identify Susceptible Locations in Medical Records via Adversarial Attacks on Deep Predictive Models](https://mlsanity.com/paper/1802.04822/) (2018)
- [Engineering Safety Requirements for Autonomous Driving with Large Language Models](https://mlsanity.com/paper/2403.16289/) (2024)
- [Safety Analysis in the Era of Large Language Models: A Case Study of STPA using ChatGPT](https://mlsanity.com/paper/2304.01246/) (2023)
- [Suspicion-Free Adversarial Attacks on Clustering Algorithms](https://mlsanity.com/paper/1911.07015/) (2019)
- [ECGadv: Generating Adversarial Electrocardiogram to Misguide Arrhythmia Classification System](https://mlsanity.com/paper/1901.03808/) (2019)
- [Understanding the One-Pixel Attack: Propagation Maps and Locality Analysis](https://mlsanity.com/paper/1902.02947/) (2019)
- [Jekyll: Attacking Medical Image Diagnostics using Deep Generative Models](https://mlsanity.com/paper/2104.02107/) (2021)
- [Adverse Events in Robotic Surgery: A Retrospective Study of 14 Years of FDA Data](https://mlsanity.com/paper/1507.03518/) (2015)
- [DKEC: Domain Knowledge Enhanced Multi-Label Classification for Diagnosis Prediction](https://mlsanity.com/paper/2310.07059/) (2023)
- [Welcome Your New AI Teammate: On Safety Analysis by Leashing Large Language Models](https://mlsanity.com/paper/2403.09565/) (2024)
- [STPA for Learning-Enabled Systems: A Survey and A New Practice](https://mlsanity.com/paper/2302.10588/) (2023)
- [Explanation-Guided Diagnosis of Machine Learning Evasion Attacks](https://mlsanity.com/paper/2106.15820/) (2021)
- [Design and Validation of an Open-Source Closed-Loop Testbed for Artificial Pancreas Systems](https://mlsanity.com/paper/2208.06479/) (2022)
- [Adversarial Artifact Detection in EEG-Based Brain-Computer Interfaces](https://mlsanity.com/paper/2212.00727/) (2022)
- [Systems-theoretic Safety Assessment of Robotic Telesurgical Systems](https://mlsanity.com/paper/1504.07135/) (2015)

## Abstract (arXiv)

The integration of AI/ML into medical devices is rapidly transforming healthcare by enhancing diagnostic and treatment facilities. However, this advancement also introduces serious cybersecurity risks due to the use of complex and often opaque models, extensive interconnectivity, interoperability with third-party peripheral devices, Internet connectivity, and vulnerabilities in the underlying technologies. These factors contribute to a broad attack surface and make threat prevention, detection, and mitigation challenging. Given the highly safety-critical nature of these devices, a cyberattack on these devices can cause the ML models to mispredict, thereby posing significant safety risks to patients. Therefore, ensuring the security of these devices from the time of design is essential. This paper underscores the urgency of addressing the cybersecurity challenges in ML-enabled medical devices at the pre-market phase. We begin by analyzing publicly available data on device recalls and adverse events, and known vulnerabilities, to understand the threat landscape of AI/ML-enabled medical devices and their repercussions on patient safety. Building on this analysis, we introduce a suite of tools and techniques designed by us to assist security analysts in conducting comprehensive premarket risk assessments. Our work aims to empower manufacturers to embed cybersecurity as a core design principle in AI/ML-enabled medical devices, thereby making them safe for patients.