Auditable Agents
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
LLM agents call tools, query databases, delegate tasks, and trigger external side effects. Once an agent system can act in the world, the question is no longer only whether harmful actions can be prevented--it is whether those actions remain answerable after deployment. We distinguish accountability (the ability to determine compliance and assign responsibility), auditability (the system property that makes accountability possible), and auditing (the process of reconstructing behavior from trustworthy evidence). Our claim is direct: no agent system can be accountable without auditability. To make this operational, we define five dimensions of agent auditability, i.e., action recoverability, lifecycle coverage, policy checkability, responsibility attribution, and evidence integrity, and identify three mechanism classes (detect, enforce, recover) whose temporal information-and-intervention constraints explain why, in practice, no single approach suffices. We support the position with layered evidence rather than a single benchmark: lower-bound ecosystem measurements suggest that even basic security prerequisites for auditability are widely unmet (617 security findings across six prominent open-source projects); runtime feasibility results show that pre-execution mediation with tamper-evident records adds only 8.3 ms median overhead; and controlled recovery experiments show that responsibility-relevant information can be partially recovered even when conventional logs are missing. We propose an Auditability Card for agent systems and identify six open research problems organized by mechanism class.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 35 of its 48 references that are in the map. First posted on arXiv 7 Apr 2026.
Builds on
35 of 48 references.
Closest ancestors · 9
Same direction, newest first
- 2026No Attacker Needed: Unintentional Cross-User Contamination in Shared-State LLM Agents1 Apr
- 2026Agent Audit: A Security Analysis System for LLM Agent Applications24 Mar
- 2026The Autonomy Tax: Defense Training Breaks LLM Agents19 Mar
- 2025Topology Matters: Measuring Memory Leakage in Multi-Agent LLMs4 Dec
- 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
- 2025AGrail: A Lifelong Agent Guardrail with Effective and Adaptive Safety Detection17 Feb
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
- 2024R-Judge: Benchmarking Safety Risk Awareness for LLM Agents18 Jan
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandbox25 Sep
Other references · 26
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2018Datasheets for Datasetscited by 1,371
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!cited by 1,106
- 2018Model Cards for Model Reportingcited by 1,068
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 956
14 more
- 2024Executable Code Actions Elicit Better LLM Agentscited by 537
- 2020Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditingcited by 380
- 2023MLAgentBench: Evaluating Language Agents on Machine Learning Experimentationcited by 339
- 2024SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decodingcited by 233
- 2023Auditing large language models: a three-layered approachcited by 135
- 2026Agent Skills for Large Language Models: Architecture, Acquisition, Security, and the Path Forwardcited by 105
- 2024Visibility into AI Agentscited by 77
- 2020Backstabber's Knife Collection: A Review of Open Source Software Supply Chain Attackscited by 57
- 2026AgentRx: Diagnosing AI Agent Failures from Execution Trajectoriescited by 53
- 2024AgentOps: Enabling Observability of LLM Agentscited by 46
- 2024AI auditing: The Broken Bus on the Road to AI Accountabilitycited by 42
- 2026Audit Trails for Accountability in Large Language Modelscited by 10
- 2026AEGIS: No Tool Call Left Unchecked -- A Pre-Execution Firewall and Audit Layer for AI Agentscited by 7
- 2026When Only the Final Text Survives: Implicit Execution Tracing for Multi-Agent Auditingcited by 2
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agentscited by 29
- 2026Proof-Carrying Agent Actions: Model-Agnostic Runtime Governance for Heterogeneous Agent Systemscited by 6
- 2026OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectoriescited by 4
- 2026CatchBench: When Can an Agent Failure Be Caught?cited by 2
- 2026Cat-DPO: Category-Adaptive Safety Alignmentcited by 1
- 2026Explanation-Bound Tool Execution for AI Agents: Server-Verified Action Claims Without Trusting Model Rationalescited by 1
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Developmentcited by 1
- 2026Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agentscited by 0
- 2026Behavioral Governance for Autonomous AI Agents: The AgentBound Frameworkcited by 0
- 2026Operational Identity: A Finite Audit of Declared and Implemented Rules of Samenesscited by 0
- 2026AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systemscited by 0
- 2026HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systemscited by 0
- 2026Janus: Evidence-Before-Effect Sagas and Offline-Verifiable Provenance for Agentic LLMs29 Sep 2026
- 2026Auditing Agent Actions through Query-Conditioned Attribution27 Sep 2026
- 2026CatchBench: When Can an Agent Failure Be Caught?24 Aug 2026
- 2026HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systems23 Aug 2026
- 2026AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systems23 Aug 2026
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development16 Aug 2026
- 2026Explanation-Bound Tool Execution for AI Agents: Server-Verified Action Claims Without Trusting Model Rationales28 Jul 2026
- 2026Operational Identity: A Finite Audit of Declared and Implemented Rules of Sameness22 Jul 2026
- 2026Behavioral Governance for Autonomous AI Agents: The AgentBound Framework29 Jun 2026
- 2026Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents9 Jun 2026
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents3 Jun 2026
- 2026Proof-Carrying Agent Actions: Model-Agnostic Runtime Governance for Heterogeneous Agent Systems2 Jun 2026
2 more
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agentscited by 29
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Developmentcited by 1
- 2026Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agentscited by 0
- 2026Auditing Agent Actions through Query-Conditioned Attributioncited by 0
- 2026Janus: Evidence-Before-Effect Sagas and Offline-Verifiable Provenance for Agentic LLMscited by 0
- 2026Janus: Evidence-Before-Effect Sagas and Offline-Verifiable Provenance for Agentic LLMs29 Sep 2026
- 2026Auditing Agent Actions through Query-Conditioned Attribution27 Sep 2026
- 2026Where Accountability Lives: Mapping Human Responsibility to Workflow Artifacts in Agentic Software Development16 Aug 2026
- 2026Trace2Policy: From Expert Behavior Traces to Self-Evolving Decision Agents9 Jun 2026
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents3 Jun 2026
- 2026Cat-DPO: Category-Adaptive Safety Alignmentcited by 1
- 2026Cat-DPO: Category-Adaptive Safety Alignment19 Apr 2026
- 2026Proof-Carrying Agent Actions: Model-Agnostic Runtime Governance for Heterogeneous Agent Systemscited by 6
- 2026OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectoriescited by 4
- 2026CatchBench: When Can an Agent Failure Be Caught?cited by 2
- 2026Explanation-Bound Tool Execution for AI Agents: Server-Verified Action Claims Without Trusting Model Rationalescited by 1
- 2026Behavioral Governance for Autonomous AI Agents: The AgentBound Frameworkcited by 0
- 2026Operational Identity: A Finite Audit of Declared and Implemented Rules of Samenesscited by 0
- 2026AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systemscited by 0
- 2026HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systemscited by 0
- 2026CatchBench: When Can an Agent Failure Be Caught?24 Aug 2026
- 2026HANSARD: A Reference Architecture for Forensic Readiness, Runtime Witnessing, and Graded Attribution in Autonomous Multi-Agent AI Systems23 Aug 2026
- 2026AUDITA: certified auditing and causal attribution of adverse outcomes in autonomous multi-agent systems23 Aug 2026
- 2026Explanation-Bound Tool Execution for AI Agents: Server-Verified Action Claims Without Trusting Model Rationales28 Jul 2026
- 2026Operational Identity: A Finite Audit of Declared and Implemented Rules of Sameness22 Jul 2026
- 2026Behavioral Governance for Autonomous AI Agents: The AgentBound Framework29 Jun 2026
- 2026Proof-Carrying Agent Actions: Model-Agnostic Runtime Governance for Heterogeneous Agent Systems2 Jun 2026
- 2026OpenClawBench: Benchmarking Process-side Anomalies in Real-world Agent Execution Trajectories28 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026When Agents Act Unwatched: The Reduced-Supervision Paradox in Agentic AI
- 2026From Traceability to Justifiability: Accountability Structures in Agentic Software Engineering
- 2025Verifiability-First Agents: Provable Observability and Lightweight Audit Agents for Controlling Autonomous LLM Systems
- 2025Traceability and Accountability in Role-Specialized Multi-Agent LLM Pipelines
- 2026AgentTrace: A Structured Logging Framework for Agent System Observability
- 2026Who Audits Whom, on What Substrate, with What Evidence? An Independence-Graded Audit Protocol for Agentic AI
- 2026Trustworthy Agentic AI: Failure Modes, Mitigation Strategies, and a Lifecycle Framework for Autonomous LLM Systems
- 2026Policy-Invisible Violations in LLM-Based Agents