Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Agentic AI systems plan, invoke tools and act with limited supervision; they are now both the subject of audits and, increasingly, the auditor. Independence, the foundation of assurance,is still applied to them as a binary. We argue that it must be graded along three orthogonal axes: principal independence (who controls the auditor), substrate independence (an auditor sharing the auditee's foundation-model family, toolchain or guardrails fails with it) and evidence independence (whether evidence is attestable rather than self-reported). Each axis has precedent; the contribution is to grade all three on a single audit, aggregate them by the weakest link, and apply the same rubric when the auditor is itself an agent. We give the model a formal basis by transplanting the beta-factor model of common-cause failure from reliability engineering, a seven-step protocol whose outputs a third party can verify, a structural detectability analysis of a procurement-controls agent audited at three grades, and a Monte Carlo study of the model in which a conventional internal audit of an agent-a real audit team, a second agent, provider logsp-surfaces 5.9% of the faults it could in principle see and none at all in half the fault classes. We map the triple to the EU AI Act as amended, ISO/IEC 42006, UK public-sector risk-management guidance and audit-regulator practice.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 23 of its 61 references that are in the map. First posted on arXiv 16 Sep 2026.
Builds on
23 of 61 references.
Closest ancestors · 4
Same direction, newest first
- 2026Steganalysis of Adaptive Covert Collusion in Tool-Using Agent Populations: A Black-Box, Cross-Principal Approach3 Aug
- 2026Agentic Witnessing: Pragmatic and Scalable TEE-Enabled Privacy-Preserving Auditing27 Apr
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents11 Oct
- 2024Secret Collusion among AI Agents: Multi-Agent Deception via Steganography12 Feb
Other references · 19
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 384
- 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
- 2023AI Control: Improving Safety Despite Intentional Subversioncited by 229
- 2024AI Agents That Mattercited by 219
- 2023Auditing large language models: a three-layered approachcited by 137
- 2023Harms from Increasingly Agentic Algorithmic Systemscited by 135
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
- 2021Algorithmic Monoculture and Social Welfarecited by 100
- 2022Picking on the Same Person: Does Algorithmic Monoculture lead to Outcome Homogenization?cited by 91
- 2025Ctrl-Z: Controlling AI Agents via Resamplingcited by 44
- 2026Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panelscited by 26
7 more
- 2026Frontier AI Auditing: Toward Rigorous Third-Party Assessment of Safety and Security Practices at Leading AI Companiescited by 11
- 2023Ecosystem-level Analysis of Deployed Machine Learning Reveals Homogeneous Outcomescited by 10
- 2026When can we trust untrusted monitoring? A safety case sketch across collusion strategiescited by 10
- 2024A Framework for Assurance Audits of Algorithmic Systemscited by 8
- 2026Governing Dynamic Capabilities: Cryptographic Binding and Reproducibility Verification for AI Agent Tool Usecited by 8
- 2023ESASCF: Expertise Extraction, Generalization and Reply Framework for an Optimized Automation of Network Security Compliancecited by 3
- 2025Subversion via Focal Points: Investigating Collusion in LLM Monitoringcited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Pramana: A Protocol-Layer Treatment of Claim Verification in Autonomous Agent Networks
- 2026When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI
- 2026Auditable Agents
- 2026BLADE: Auditing Information Use for Failure Attribution in LLM Multi-Agent Systems
- 2026CrossAudit: A Git-Native, Cross-Vendor Audit Loop for Agentic Science
- 2026Auditing Discovery Claims: A Two-Sided Criterion for Agentic Science, with the Negative Side Decidable
- 2026Governed Auditable Decisioning Under Uncertainty: Synthesis and Agentic Extension
- 2026Governing Agentic AI in FinTech