# Threat Modeling for AI: The Case for an Asset-Centric Approach

arXiv: https://arxiv.org/abs/2505.06315 · HTML: https://mlsanity.com/paper/2505.06315/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 54 references that are in the map.
- First posted on arXiv 8 May 2025.

## Builds on (most cited first, 18 of 18)

- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Label-Only Membership Inference Attacks](https://mlsanity.com/paper/2007.14321/) (2020)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Poisoning Web-Scale Training Datasets is Practical](https://mlsanity.com/paper/2302.10149/) (2023)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [Data-Free Model Extraction](https://mlsanity.com/paper/2011.14779/) (2020)
- [Backdoor Attacks on Pre-trained Models by Layerwise Weight Poisoning](https://mlsanity.com/paper/2108.13888/) (2021)
- [Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search](https://mlsanity.com/paper/1903.12269/) (2019)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures](https://mlsanity.com/paper/1808.04761/) (2018)
- [Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks](https://mlsanity.com/paper/1906.01017/) (2019)
- [DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips](https://mlsanity.com/paper/2003.13746/) (2020)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [Architectural Backdoors in Neural Networks](https://mlsanity.com/paper/2206.07840/) (2022)
- [HarDNN: Feature Map Vulnerability Evaluation in CNNs](https://mlsanity.com/paper/2002.09786/) (2020)
- [Atlas: A Framework for ML Lifecycle Provenance & Transparency](https://mlsanity.com/paper/2502.19567/) (2025)

## Built on by (most cited first, 1 of 1)

- [A Survey of Vibe Coding with Large Language Models](https://mlsanity.com/paper/2510.12399/) (2025)

## Abstract (arXiv)

Recent advances in AI are transforming AI's ubiquitous presence in our world from that of standalone AI-applications into deeply integrated AI-agents. These changes have been driven by agents' increasing capability to autonomously make decisions and initiate actions, using existing applications; whether those applications are AI-based or not. This evolution enables unprecedented levels of AI integration, with agents now able to take actions on behalf of systems and users -- including, in some cases, the powerful ability for the AI to write and execute scripts as it deems necessary. With AI systems now able to autonomously execute code, interact with external systems, and operate without human oversight, traditional security approaches fall short. This paper introduces an asset-centric methodology for threat modeling AI systems that addresses the unique security challenges posed by integrated AI agents. Unlike existing top-down frameworks that analyze individual attacks within specific product contexts, our bottom-up approach enables defenders to systematically identify how vulnerabilities -- both conventional and AI-specific -- impact critical AI assets across distributed infrastructures used to develop and deploy these agents. This methodology allows security teams to: (1) perform comprehensive analysis that communicates effectively across technical domains, (2) quantify security assumptions about third-party AI components without requiring visibility into their implementation, and (3) holistically identify AI-based vulnerabilities relevant to their specific product context. This approach is particularly relevant for securing agentic systems with complex autonomous capabilities. By focusing on assets rather than attacks, our approach scales with the rapidly evolving threat landscape while accommodating increasingly complex and distributed AI development pipelines.