mlsanityData to 7 Oct 2026

LLM-enabled Applications Require System-Level Threat Monitoring

arXiv 2602.19844 ↗First posted 23 Feb 2026cs.CR cs.AI cs.SEcited by 2

Not on the map. Why?

Abstract (arXiv)

LLM-enabled applications are rapidly reshaping the software ecosystem by using large language models as core reasoning components for complex task execution. This paradigm shift, however, introduces fundamentally new reliability challenges and significantly expands the security attack surface, due to the non-deterministic, learning-driven, and difficult-to-verify nature of LLM behavior. In light of these emerging and unavoidable safety challenges, we argue that such risks should be treated as expected operational conditions rather than exceptional events, necessitating a dedicated incident-response perspective. Consequently, the primary barrier to trustworthy deployment is not further improving model capability but establishing system-level threat monitoring mechanisms that can detect and contextualize security-relevant anomalies after deployment -- an aspect largely underexplored beyond testing or guardrail-based defenses. Accordingly, this position paper advocates systematic and comprehensive monitoring of security threats in LLM-enabled applications as a prerequisite for reliable operation and a foundation for dedicated incident-response frameworks.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 75 of its 109 references that are in the map. First posted on arXiv 23 Feb 2026.

Builds on

75 of 109 references.

References · 75

  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  5. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  6. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 494
  7. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 402
  8. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  9. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  10. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 329
  11. 2023Poisoning Language Models During Instruction Tuningcited by 254
  12. 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 230
38 more
  1. 2021Adversarial GLUE: A Multi-Task Benchmark for Robustness Evaluation of Language Modelscited by 227
  2. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  3. 2025Why Language Models Hallucinatecited by 205
  4. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agentscited by 196
  5. 2017Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimizationcited by 188
  6. 2023Poisoning Retrieval Corpora by Injecting Adversarial Passagescited by 153
  7. 2024Stealing Part of a Production Language Modelcited by 152
  8. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  9. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoningcited by 144
  10. 2024The Emerged Security and Privacy of LLM Agent: A Survey with Case Studiescited by 127
  11. 2025Memory Injection Attacks on LLM Agents via Query-Only Interactioncited by 127
  12. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 126
  13. 2025DataSentinel: A Game-Theoretic Detection of Prompt Injection Attackscited by 124
  14. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  15. 2025A Survey of Context Engineering for Large Language Modelscited by 120
  16. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  17. 2024GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoningcited by 113
  18. 2025Unveiling Privacy Risks in LLM Agent Memorycited by 99
  19. 2024Systematic Biases in LLM Simulations of Debatescited by 98
  20. 2025PromptArmor: Simple yet Effective Prompt Injection Defensescited by 98
  21. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agentscited by 89
  22. 2020Policy Teaching via Environment Poisoning: Training-time Adversarial Attacks against Reinforcement Learningcited by 81
  23. 2024LazyLLM: Dynamic Token Pruning for Efficient Long Context LLM Inferencecited by 70
  24. 2020Subpopulation Data Poisoning Attackscited by 68
  25. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedbackcited by 64
  26. 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Serverscited by 63
  27. 2023Latent Jailbreak: A Benchmark for Evaluating Text Safety and Output Robustness of Large Language Modelscited by 56
  28. 2024Towards Implicit Bias Detection and Mitigation in Multi-Agent LLM Interactionscited by 50
  29. 2023Spear Phishing With Large Language Modelscited by 49
  30. 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agentscited by 46
  31. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agentscited by 42
  32. 2025AgentArmor: Enforcing Program Analysis on Agent Runtime Trace to Defend Against Prompt Injectioncited by 39
  33. 2024Membership Inference Attacks Against In-Context Learningcited by 38
  34. 2024Denial-of-Service Poisoning Attacks against Large Language Modelscited by 34
  35. 2025Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreakingcited by 34
  36. 2024Crabs: Consuming Resource via Auto-generation for LLM-DoS Attack under Black-box Settingscited by 33
  37. 2025LLMs Cannot Reliably Judge (Yet?): A Comprehensive Assessment on the Robustness of LLM-as-a-Judgecited by 33
  38. 2024Defense Against Prompt Injection Attack by Leveraging Attack Techniquescited by 31

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Inverting the Shield: Systematically Generating Safety Tests from Policy Specificationscited by 0
  2. 2026Safe Actions Alone Do Not Ensure Safe Agents: Identifying Unfulfilled Obligations with Guard Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Uncovering Vulnerabilities of LLM-Assisted Cyber Threat Intelligence
  2. 2025Bridging AI and Software Security: A Comparative Vulnerability Assessment of LLM Agent Deployment Paradigms
  3. 2026Antaeus: Hunting Repository-Level Logic Vulnerabilities via Context-Grounded LLM Reasoning ↗
  4. 2024BELLS: A Framework Towards Future Proof Benchmarks for the Evaluation of LLM Safeguards
  5. 2026Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios
  6. 2024A New Era in LLM Security: Exploring Security Concerns in Real-World LLM-based Systems
  7. 2026Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis
  8. 2025VulInstruct: Teaching LLMs Root-Cause Reasoning for Vulnerability Detection via Security Specifications