mlsanityData to 14 Oct 2026

PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents

arXiv 2606.29225 ↗First posted 28 Jun 2026cs.AI cs.CLcited by 6

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM agents handle user requests on behalf of organizations through tool calls and must follow the company policies stated in their system prompts. Prior work approaches this as a safeguarding problem -- external checks that block non-compliant agent actions. We argue that policy adherence is a broader problem: real workflows unfold across many turns, require explicit user confirmation and prerequisite reads, and hinge on the content of the dialogue rather than on any single argument value. Meeting this bar requires (i) full conversation context, (ii) self-reasoning over the policy and the current dialogue, and (iii) conversation-specific remediation that guides the agent's next turn -- three capabilities that prior safeguard work has often underestimated. We introduce POLICYGUARD, a sub-agent verifier that shares the agent's view of the dialogue, reasons over the policy in context, and provides actionable feedback for the agent's next turn. On tau^2-BENCH airline across three vendors (GPT-5.4, Claude Sonnet 4.6, Gemini 2.5 Pro) with four trials per setting, POLICYGUARD improves PASS4 by +12.0 / +6.0 / +12.0 pp. Per-call analyses show POLICYGUARD achieves higher policy-violation recall while blocking roughly half as often as argument-level guards.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 18 of its 25 references that are in the map. First posted on arXiv 28 Jun 2026.

Builds on

18 of 25 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Solver-Aided Verification of Policy Compliance in Tool-Augmented LLM Agents20 Mar
  2. 2026ToolSafe: Enhancing Tool Invocation Safety of LLM-based agents via Proactive Step-level Guardrail and Feedback15 Jan
  3. 2025Towards Enforcing Company Policy Adherence in Agentic Workflows22 Jul
  4. 2025Effective Red-Teaming of Policy-Adherent Agents11 Jun
  5. 2025ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning26 Mar
  6. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar

Other references · 12

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,443
  7. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
  8. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365
  11. 2024CRMArena: Understanding the Capacity of LLM Agents to Perform Professional CRM Tasks in Realistic Environmentscited by 71
  12. 2026Near-Miss: Latent Policy Failure Detection in Agentic Workflowscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026PolicyBank: Evolving Policy Understanding for LLM Agents
  2. 2026Beyond IVR: Benchmarking Customer Support LLM Agents for Business-Adherence
  3. 2026PA3: Policy-Aware Agent Alignment through Chain-of-Thought
  4. 2026AutoCedar: An Agentic Framework for Verifier-Guided Access Control Policy Synthesis
  5. 2026From Natural Language Policies to Executable Obligations: A Verification Harness for Dependable In-Car LLM Agents
  6. 2026PolicyGuard: Prompt-Configurable Semantic DLP for LLM Coding Agents
  7. 2026Policy-Invisible Violations in LLM-Based Agents
  8. 2026ActGov: Governing LLM Agent Actions via Policy-Constrained Validation