mlsanityData to 14 Oct 2026

Runtime Compliance Verification for AI Agents

arXiv 2606.19242 ↗First posted 17 Jun 2026cs.SEcited by 1

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

AI agents now handle personal data through tool use, function calls, and multi turn dialogue, which can create obligations under the General Data Protection Regulation (GDPR). Current testing practices mainly rely on offline red teaming or static prompt review, but they do not guarantee at runtime that agent behavior follows regulatory rules. We propose C-Trace (Compliance Trace based Runtime Agent Conformance Enforcement), a verification framework that: (i) expresses a subset of GDPR requirements, including consent, purpose limitation, data minimization, and the right to erasure, as formal policy predicates over agent execution traces; (ii) uses a runtime monitor that intercepts every tool invocation and model output and rejects non-compliant actions; and (iii) tests the agent with attack dialogues, including DSPy generated prompts and verbatim prompts from red teaming corpora, that try to induce violations. We evaluate the framework on four case studies reframed to GDPR. Under 10 percent per-category extractor noise, including drop-out and over-typing, the monitor keeps the attack success rate at less than or equal to 12 percent, below the baselines we compare against, and false positives at less than or equal to 16 percent, and reaches 0 percent ASR under perfect extraction.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 9 of its 22 references that are in the map. First posted on arXiv 17 Jun 2026.

Builds on

9 of 22 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal6 Feb
  2. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection23 Feb
  3. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learned23 Aug
  4. 2022Red Teaming Language Models with Language Models7 Feb

Other references · 5

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  4. 2023"Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Modelscited by 489
  5. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 365

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025AudAgent: Automated Auditing of Privacy Policy Compliance in AI Agents
  2. 2026One Monitor Does Not Fit All: Formalism-Agnostic Runtime Verification for Agentic AI
  3. 2026AgentSpy: Making AI Agent Behavior Observable
  4. 2026AgentTrust: Runtime Safety Evaluation and Interception for AI Agent Tool Use
  5. 2026CAVA: Canonical Action Verification and Attestation for Runtime Governance of Agentic AI Systems
  6. 2026AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents
  7. 2026Silence Is Endorsement: Verification-Status Laundering in LLM Agent Pipelines
  8. 2026VIGIL: Runtime Enforcement of Behavioral Specifications in AI Agent Skills