# A Framework for Rapidly Developing and Deploying Protection Against Large Language Model Attacks

arXiv: https://arxiv.org/abs/2509.20639 · HTML: https://mlsanity.com/paper/2509.20639/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 44 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 15 of 15)

- [Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection](https://mlsanity.com/paper/2302.12173/) (2023)
- [Baseline Defenses for Adversarial Attacks Against Aligned Language Models](https://mlsanity.com/paper/2309.00614/) (2023)
- [Formalizing and Benchmarking Prompt Injection Attacks and Defenses](https://mlsanity.com/paper/2310.12815/) (2023)
- [Defeating Prompt Injections by Design](https://mlsanity.com/paper/2503.18813/) (2025)
- [Certifying LLM Safety against Adversarial Prompting](https://mlsanity.com/paper/2309.02705/) (2023)
- [Attention Tracker: Detecting Prompt Injection Attacks in LLMs](https://mlsanity.com/paper/2411.00348/) (2024)
- [Identifying and Mitigating Vulnerabilities in LLM-Integrated Applications](https://mlsanity.com/paper/2311.16153/) (2023)
- [PromptShield: Deployable Detection for Prompt Injection Attacks](https://mlsanity.com/paper/2501.15145/) (2025)
- [Bypassing LLM Guardrails: An Empirical Analysis of Evasion Attacks against Prompt Injection and Jailbreak Detection Systems](https://mlsanity.com/paper/2504.11168/) (2025)
- [Lessons From Red Teaming 100 Generative AI Products](https://mlsanity.com/paper/2501.07238/) (2025)
- [Rapid Response: Mitigating LLM Jailbreaks with a Few Examples](https://mlsanity.com/paper/2411.07494/) (2024)
- [Adversarial Machine Learning: Bayesian Perspectives](https://mlsanity.com/paper/2003.03546/) (2020)
- [Wisdom of the Ensemble: Improving Consistency of Deep Learning Models](https://mlsanity.com/paper/2011.06796/) (2020)
- [Unraveling the Key Components of OOD Generalization via Diversification](https://mlsanity.com/paper/2312.16313/) (2023)
- [Enhancing Security in LLM Applications: A Performance Evaluation of Early Detection Systems](https://mlsanity.com/paper/2506.19109/) (2025)

## Built on by (most cited first, 3 of 3)

- [Machine Learning-Based Detection of MCP Attacks](https://mlsanity.com/paper/2604.10534/) (2026)
- [Improving Labeling Consistency with Detailed Constitutional Definitions and AI-Driven Evaluation](https://mlsanity.com/paper/2605.24247/) (2026)
- [PatchOptic for Shared-State LLM Workflows with Projected Views and Verified Structured Updates](https://mlsanity.com/paper/2607.05483/) (2026)

## Abstract (arXiv)

The widespread adoption of Large Language Models (LLMs) has revolutionized AI deployment, enabling autonomous and semi-autonomous applications across industries through intuitive language interfaces and continuous improvements in model development. However, the attendant increase in autonomy and expansion of access permissions among AI applications also make these systems compelling targets for malicious attacks. Their inherent susceptibility to security flaws necessitates robust defenses, yet no known approaches can prevent zero-day or novel attacks against LLMs. This places AI protection systems in a category similar to established malware protection systems: rather than providing guaranteed immunity, they minimize risk through enhanced observability, multi-layered defense, and rapid threat response, supported by a threat intelligence function designed specifically for AI-related threats. Prior work on LLM protection has largely evaluated individual detection models rather than end-to-end systems designed for continuous, rapid adaptation to a changing threat landscape. We present a production-grade defense system rooted in established malware detection and threat intelligence practices. Our platform integrates three components: a threat intelligence system that turns emerging threats into protections; a data platform that aggregates and enriches information while providing observability, monitoring, and ML operations; and a release platform enabling safe, rapid detection updates without disrupting customer workflows. Together, these components deliver layered protection against evolving LLM threats while generating training data for continuous model improvement and deploying updates without interrupting production.