Threat Modeling for AI: The Case for an Asset-Centric Approach

arXiv 2505.06315 ↗First posted 8 May 2025cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advances in AI are transforming AI's ubiquitous presence in our world from that of standalone AI-applications into deeply integrated AI-agents. These changes have been driven by agents' increasing capability to autonomously make decisions and initiate actions, using existing applications; whether those applications are AI-based or not. This evolution enables unprecedented levels of AI integration, with agents now able to take actions on behalf of systems and users -- including, in some cases, the powerful ability for the AI to write and execute scripts as it deems necessary. With AI systems now able to autonomously execute code, interact with external systems, and operate without human oversight, traditional security approaches fall short. This paper introduces an asset-centric methodology for threat modeling AI systems that addresses the unique security challenges posed by integrated AI agents. Unlike existing top-down frameworks that analyze individual attacks within specific product contexts, our bottom-up approach enables defenders to systematically identify how vulnerabilities -- both conventional and AI-specific -- impact critical AI assets across distributed infrastructures used to develop and deploy these agents. This methodology allows security teams to: (1) perform comprehensive analysis that communicates effectively across technical domains, (2) quantify security assumptions about third-party AI components without requiring visibility into their implementation, and (3) holistically identify AI-based vulnerabilities relevant to their specific product context. This approach is particularly relevant for securing agentic systems with complex autonomous capabilities. By focusing on assets rather than attacks, our approach scales with the rapidly evolving threat landscape while accommodating increasingly complex and distributed AI development pipelines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 54 references that are in the map. First posted on arXiv 8 May 2025.

Builds on

18 of 54 references.

References · 18

2020Extracting Training Data from Large Language Modelscited by 2,189
2016Understanding deep learning requires rethinking generalizationcited by 1,674
2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
2020Label-Only Membership Inference Attackscited by 388
2020Weight Poisoning Attacks on Pre-trained Modelscited by 341
2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
2020A Survey on the Explainability of Supervised Machine Learningcited by 227
2020Data-Free Model Extractioncited by 124
2021Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoningcited by 113
2019Bit-Flip Attack: Crushing Neural Network with Progressive Bit Searchcited by 108
2023Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Modelscited by 75
2018Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architecturescited by 57

6 more

2019Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attackscited by 57
2020DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flipscited by 56
2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
2022Architectural Backdoors in Neural Networkscited by 25
2020HarDNN: Feature Map Vulnerability Evaluation in CNNscited by 16
2025Atlas: A Framework for ML Lifecycle Provenance & Transparencycited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way