Correct Answers, Unsupported Findings: Evidence Binding in Forensic Reconstruction of LLM Agent Logs
New in LLM agents and tool use; no direction yet. LLM agents and tool use
Abstract (arXiv)
Forensic reconstruction of LLM-agent actions requires not only recovering the correct value, but establishing which preserved record supports that finding. Tool logs, generated explanations, and local citation identifiers capture different parts of this evidence, yet a citation identifier does not establish a source unless its binding to a record is preserved. We audit this distinction using 64 mechanically checkable cases from saved AgentDojo Banking executions. Two LLM readers reconstruct source relationships under controlled variations in visible evidence and identifier-to-record bindings. We separately evaluate complete-record agreement, evidence-grounded findings, justified abstention, and unsupported assertions. With original identifiers and no binding table, Sonnet recovered every literal source location but made unsupported citation-source assertions in 26 of 28 cases requiring the missing relation; 22 nevertheless matched the complete reference. Adding explicit bindings improved grounded reconstruction for both readers, whereas identifier renaming alone provided no consistent remedy. A deterministic same-packet comparator correctly resolved the bounded task or abstained throughout. These results show that factual agreement alone is insufficient for evaluating forensic reconstruction of agent logs and motivate preserving explicit record bindings to distinguish supported findings from correct guesses.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 0 of its 30 references that are in the map. First posted on arXiv 7 Oct 2026.
Builds on
0 of 30 references.
No references here yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026From Agent Traces to Trust: A Survey of Evidence Tracing and Execution Provenance in LLM Agents
- 2026Agentic Data Cleaning Without a Clean Reference: An Experimental Study of Capabilities and Trade-offs
- 2026When Evidence Changes: Evaluating Memory Repair and Re-reading in Language-Model Agents
- 2026Reliable Post-Retrieval Assembly for Agent Memory: Separating Evidence Extraction from Policy Execution
- 2026Memory Is a Derivation: The Distributed-Evidence Paradox in Long-Term Agents
- 2026Auditing Provenance Sensitivity in LLM Agent Action Selection
- 2026ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents
- 2025EMULATE: A Multi-Agent Framework for Determining the Veracity of Atomic Claims by Emulating Human Actions