mlsanityData to 7 Oct 2026

Securing AI Agents with Information-Flow Control

arXiv 2505.23643 ↗First posted 29 May 2025cs.CR cs.AIcited by 108

Not on the map. Why?

Cited by 71 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

As AI agents become increasingly autonomous and capable, ensuring their security against vulnerabilities such as prompt injection becomes critical. This paper explores the use of information-flow control (IFC) to provide security guarantees for AI agents. We present a formal model to reason about the security and expressiveness of agent planners. Using this model, we characterize the class of properties enforceable by dynamic taint-tracking and construct a taxonomy of tasks to evaluate security and utility trade-offs of planner designs. Informed by this exploration, we present Fides, a planner that tracks confidentiality and integrity labels, deterministically enforces security policies, and introduces novel primitives for selectively hiding information. Its evaluation in AgentDojo demonstrates that this approach enables us to complete a broad range of tasks with security guarantees. A tutorial to walk readers through the the concepts introduced in the paper can be found at https://github.com/microsoft/fides

Key facts. Cited by 108 papers in the map (data through 7 Oct 2026). Builds on 25 of its 44 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

25 of 44 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  4. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,906
  5. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  6. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  7. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agentscited by 476
  8. 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 441
  9. 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agentscited by 341
  10. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  11. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  12. 2025Defeating Prompt Injections by Designcited by 256
13 more
  1. 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 237
  2. 2024Defending Against Indirect Prompt Injection Attacks With Spotlightingcited by 233
  3. 2024Magentic-One: A Generalist Multi-Agent System for Solving Complex Taskscited by 221
  4. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  5. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  6. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 99
  7. 2024System-Level Defense against Indirect Prompt Injection Attacks: An Information Flow Control Perspectivecited by 91
  8. 2024The Task Shield: Enforcing Task Alignment to Defend Against Indirect Prompt Injection in LLM Agentscited by 89
  9. 2024GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysiscited by 87
  10. 2025RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakagecited by 64
  11. 2024Instructional Segment Embedding: Improving LLM Safety with Instruction Hierarchycited by 62
  12. 2024Get my drift? Catching LLM Task Drift with Activation Deltascited by 61
  13. 2024Embedding-based classifiers can detect prompt injection attackscited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 88.

108 papers.

Sort:

By area:

  1. 2025The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses Against Llm Jailbreaks and Prompt Injectionscited by 108
  2. 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attackscited by 82
  3. 2026Towards Verifiably Safe Tool Use for LLM Agentscited by 34
  4. 2026Formal Policy Enforcement for Real-World Agentic Systemscited by 31
  5. 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agentscited by 29
  6. 2026The Landscape of Prompt Injection Threats in LLM Agents: From Taxonomy to Analysiscited by 25
  7. 2025PromptLocate: Localizing Prompt Injection Attackscited by 23
  8. 2025BrowseSafe: Understanding and Preventing Prompt Injection Within AI Browser Agentscited by 23
  9. 2025ObliInjection: Order-Oblivious Prompt Injection Attack to LLM Agents with Multi-source Datacited by 17
  10. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 15
  11. 2025ceLLMate: Sandboxing Browser AI Agentscited by 14
  12. 2026Ghost in the Agent: Redefining Information Flow Tracking for LLM Agentscited by 14
38 more
  1. 2026MemLineage: Lineage-Guided Enforcement for LLM Agent Memorycited by 14
  2. 2026Securing LLM-Agent Long-Term Memory Against Poisoning: Non-Malleable, Origin-Bound Authority with Machine-Checked Guaranteescited by 14
  3. 2025PIShield: Detecting Prompt Injection Attacks via Intrinsic LLM Featurescited by 13
  4. 2026PISmith: Reinforcement Learning-based Red Teaming for Prompt Injection Defensescited by 13
  5. 2026Agent-Sentry: Bounding LLM Agents via Execution Provenancecited by 13
  6. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 13
  7. 2025CommandSans: Securing AI Agents with Surgical Precision Prompt Sanitizationcited by 12
  8. 2025Breaking and Fixing Defenses Against Control-Flow Hijacking in Multi-Agent Systemscited by 12
  9. 2026Aligning Provenance with Authorization: A Dual-Graph Defense for LLM Agentscited by 12
  10. 2025How Not to Detect Prompt Injections with an LLMcited by 11
  11. 2025SoK: Trust-Authorization Mismatch in LLM Agent Interactionscited by 11
  12. 2026ChainCaps: Composition-Safe Tool-Using Agents via Monotonic Capability Attenuationcited by 11
  13. 2025SecInfer: Preventing Prompt Injection via Inference-time Scalingcited by 10
  14. 2026Optimizing Agent Planning for Security and Autonomycited by 10
  15. 2025Systems Security Foundations for Agentic Computingcited by 9
  16. 2026AgentGuardian: Learning Access Control Policies to Govern AI Agent Behaviorcited by 9
  17. 2026AI Agents May Always Fall for Prompt Injectionscited by 9
  18. 2025Preventing Prompt Injection with Type-Directed Privilege Separationcited by 8
  19. 2026The Granularity Mismatch in Agent Security: Argument-Level Provenance Solves Enforcement and Isolates the LLM Reasoning Bottleneckcited by 8
  20. 2026Reframing LLM Agent Security as an Agent-Human Interaction Problemcited by 8
  21. 2026Agent libOS: A Runtime Substrate for Capability-Controlled Self-Evolving LLM Agentscited by 8
  22. 2026VIGIL: Runtime Enforcement of Behavioral Specifications in AI Agent Skillscited by 8
  23. 2025When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agentscited by 7
  24. 2025Taxonomy, Evaluation and Exploitation of IPI-Centric LLM Agent Defense Frameworkscited by 7
  25. 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malwarecited by 6
  26. 2025AttnTrace: Contextual Attribution of Prompt Injection and Knowledge Corruptioncited by 5
  27. 2026AC4A: Access Control for Agentscited by 5
  28. 2026AID-Guard: Stateful Authorization for Delegated Agent Effectscited by 5
  29. 2026Semantic Intent Fragmentation: A Single-Shot Compositional Attack on Multi-Agent AI Pipelinescited by 4
  30. 2025Can LLMs Make (Personalized) Access Control Decisions?cited by 3
  31. 2026The LLMbda Calculus: AI Agents, Conversations, and Information Flowcited by 3
  32. 2026An AI Agent Execution Environment to Safeguard User Datacited by 3
  33. 2026Enforcing Benign Trajectories: A Behavioral Firewall for Structured-Workflow AI Agentscited by 3
  34. 2026Web Agents Should Adopt the Plan-Then-Execute Paradigmcited by 3
  35. 2026Boiling the Frog: A Multi-Turn Benchmark for Agentic Safetycited by 3
  36. 2026Untrusted Content Masking for Web Agents with Security Guaranteescited by 3
  37. 2026Position: Capability Control Should be a Separate Goal From Alignmentcited by 2
  38. 2026Securing LLM Agents Need Intent-to-Execution Integritycited by 2

50 most cited of 108 All 108 (JSON)