Ghost in the Context: Policy-Carriage Integrity in LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents choose actions from bounded decision states assembled from system policy, runtime state, tools, workload content, and the final request. We study policy-carriage integrity: applicable trusted policies must remain present, sound, and correctly bound in the decision state immediately before action. Under controlled pressure replay over AutoGen/tau3 and OpenHands/SWE-bench traces, the tested protected-placement configurations preserve policy across the pressure sweep, while task-local placement exhibits eviction, weakening, or over-budget continuation depending on the context manager. We keep this result state-level: a fixed-assembler behavioral calibration produced 0/90 unsafe-action proposals and 0/90 unguarded policy violations, so policy absence alone did not establish unsafe model behavior. The resulting design guidance is systems-level: assign typed provenance to policy state, isolate control budget, check before assembly that the complete active policy set fits, fail closed on overload, and enforce structured policies at the action boundary. We present ControlCapsule as a reference design pattern for these requirements; exact active-policy replay + preflight remains the key baseline.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 40 of its 51 references that are in the map. First posted on arXiv 2 May 2026.
Builds on
40 of 51 references.
Closest ancestors · 11
Same direction, newest first
- 2026Zombie Agents: Persistent Control of Self-Evolving LLM Agents via Self-Reinforcing Injections17 Feb
- 2026AgentSys: Secure and Dynamic LLM Agents Through Explicit Hierarchical Memory Management7 Feb
- 2025MemoryGraft: Persistent Compromise of LLM Agents via Poisoned Experience Retrieval18 Dec
- 2025A-MemGuard: A Proactive Defense Framework for LLM-Based Agent Memory29 Sep
- 2025LLM Agents Should Employ Security Principles29 May
- 2025Memory Injection Attacks on LLM Agents via Query-Only Interaction5 Mar
- 2025Unveiling Privacy Risks in LLM Agent Memory17 Feb
- 2024Agent Security Bench (ASB): Formalizing and Benchmarking Attacks and Defenses in LLM-based Agents3 Oct
- 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
- 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 29
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2023MemGPT: Towards LLMs as Operating Systemscited by 1,389
- 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
17 more
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
- 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
- 2024The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructionscited by 493
- 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
- 2023Benchmarking and Defending Against Indirect Prompt Injection Attacks on Large Language Modelscited by 264
- 2023ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understandingcited by 176
- 2022SCROLLS: Standardized CompaRison Over Long Language Sequencescited by 170
- 2025Memory OS of AI Agentcited by 150
- 2024In-Context Learning with Long-Context Models: An In-Depth Explorationcited by 128
- 2025NoLiMa: Long-Context Evaluation Beyond Literal Matchingcited by 126
- 2025MemInsight: Autonomous Memory Augmentation for LLM Agentscited by 85
- 2025MemBench: Towards More Comprehensive Evaluation on the Memory of LLM-based Agentscited by 84
- 2024LV-Eval: A Balanced Long-Context Benchmark with 5 Length Levels Up to 256Kcited by 81
- 2025Memory as Action: Autonomous Context Curation for Long-Horizon Agentic Taskscited by 66
- 2025Defending Against Prompt Injection with DataFiltercited by 35
- 2026Adaptive Memory Admission Control for LLM Agentscited by 23
- 2026Memex(RL): Scaling Long-Horizon LLM Agents via Indexed Experience Memorycited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reason Less, Verify More: Deterministic Gates Recover a Silent Policy-Violation Failure Mode in Tool-Using LLM Agents
- 2026Stale Does Not Mean Unsafe: Guard Precision for Tool-Using LLM Agents under Infrastructure State Races
- 2026VEPA: Verified Evolving Formal Policy for Lifelong LLM Agent Guardrails
- 2026AgentFlow: A Flow-Centric Policy Language and Framework for Securing LLM Agent Systems
- 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validation
- 2026CONTINUITY: Security-Context Contracts for Composable LLM Agent Controls
- 2026From Natural Language Policies to Executable Obligations: A Verification Harness for Dependable In-Car LLM Agents
- 2026Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents