PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents handle user requests on behalf of organizations through tool calls and must follow the company policies stated in their system prompts. Prior work approaches this as a safeguarding problem -- external checks that block non-compliant agent actions. We argue that policy adherence is a broader problem: real workflows unfold across many turns, require explicit user confirmation and prerequisite reads, and hinge on the content of the dialogue rather than on any single argument value. Meeting this bar requires (i) full conversation context, (ii) self-reasoning over the policy and the current dialogue, and (iii) conversation-specific remediation that guides the agent's next turn -- three capabilities that prior safeguard work has often underestimated. We introduce POLICYGUARD, a sub-agent verifier that shares the agent's view of the dialogue, reasons over the policy in context, and provides actionable feedback for the agent's next turn. On tau^2-BENCH airline across three vendors (GPT-5.4, Claude Sonnet 4.6, Gemini 2.5 Pro) with four trials per setting, POLICYGUARD improves PASS4 by +12.0 / +6.0 / +12.0 pp. Per-call analyses show POLICYGUARD achieves higher policy-violation recall while blocking roughly half as often as argument-level guards.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 18 of its 25 references that are in the map. First posted on arXiv 28 Jun 2026.
Builds on
18 of 25 references.
Closest ancestors · 6
Same direction, newest first
- 2026Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agents20 Mar
- 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
- 2025Towards Enforcing Company Policy Adherence in Agentic Workflows22 Jul
- 2025Effective Red-Teaming of Policy-Adherent Agents11 Jun
- 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
Other references · 12
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
- 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 71
- 2026Near-Miss: Latent Policy Failure Detection in Agentic Workflowscited by 4
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM Agentscited by 3
- 2026STAGE: Stateful Translation to Agentic Graph Execution with Policy-Scoped Context and Deterministic Controlcited by 0
- 2026Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructionscited by 0
- 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agentscited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agentscited by 0
- 2026Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collectioncited by 0
- 2026NOMOS: Compiling Written Policies into Statically Verified Tool-Call Gates for LLM Agents8 Oct 2026
- 2026Following the Flow the Law Requires: Training Voice Agents to Transfer to Unseen Regulated Procedures in Debt Collection1 Oct 2026
- 2026What Can Component-Replacement Evidence Establish? A Critical Scoping Review of Local Decisions in LLM Agents30 Sep 2026
- 2026Authorization Closure Graph: Minimal Repair for LLM Agents with Evolving User Instructions26 Sep 2026
- 2026STAGE: Stateful Translation to Agentic Graph Execution with Policy-Scoped Context and Deterministic Control23 Aug 2026
- 2026PolicyGuide: From Guarding One Action to Guiding the Whole Workflow for Policy-Compliant LLM Agents20 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026PolicyBank: Evolving Policy Understanding for LLM Agents
- 2026Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence
- 2026PA3: Policy-Aware Agent Alignment through Chain-of-Thought
- 2026AutoCedar: An Agentic Framework for Verifier-Guided Access Control Policy Synthesis
- 2026From Natural Language Policies to Executable Obligations: A Verification Harness for Dependable In-Car LLM Agents
- 2026PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents
- 2026Policy-Invisible Violations in LLM-Based Agents
- 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validation