ClaimReceipt: Verifying Evidence Sufficiency and Coverage in Agent Evaluations
Not on the map. Why?
Abstract (arXiv)
Agent evaluations face two distinct evidentiary questions: whether a reported claim is recomputable from retained evidence (sufficiency), and whether the retained records cover the committed experiment set (coverage). Generic logs and hash-linked transcripts answer neither reliably. We introduce ClaimReceipt, a claim-relative receipt specification and selective verifier that binds typed transaction evidence to a signed experiment manifest and returns PASS, INVALID, or INCONCLUSIVE per claim. We freeze the specification before implementation (SHA-256 18d109...b81). On 1,392 historical buyer--seller records, a CR-2 verifier reproduces all five manually labeled audit verdicts, exactly replays 600 deterministic and 792 post-generation records, makes every one of 13 declared field groups non-redundant under tested ablations, and returns the expected result on 11/11 semantic faults with 0/8 false positives. We then run a separate prospective CR-3 epoch: 30 assignments are committed before inference, terminal receipts are signed and chained, and private evidence is encrypted for an auditor. Complete evidence yields coverage and accounting PASS; withholding one terminal receipt returns INCONCLUSIVE_COVERAGE, while withholding all private openings preserves coverage and protocol verification but makes economic claims inconclusive, exactly matching a preregistered prediction. Receipt instrumentation adds 0.021% of model-inference time and 9.9 KB per transaction. A specification-legibility probe indicates that our own frozen specification is not yet unambiguous to an independent reader. Claim verification therefore requires both claim-sufficient evidence and a committed universe against which omissions become visible.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 10 of its 21 references that are in the map. First posted on arXiv 2 Sep 2026.
Builds on
10 of 21 references.
References · 10
- 2024Qwen2.5 Technical Reportcited by 4,942
- 2018Datasheets for Datasetscited by 1,386
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2018Model Cards for Model Reportingcited by 1,080
- 2019Measurement and Fairnesscited by 308
- 2024AI Agents That Mattercited by 219
- 2014An introduction to Docker for reproducible research, with examples from the R environmentcited by 26
- 2026When Outcome Looks Right But Discipline Fails: Trace-Based Evaluation Under Hidden Competitor Statecited by 2
- 2026When Aggregate Alignment Misleads: Auditing Policy Repair Without Per-State Expert Actionscited by 2
- 2026When Guardrails Look Effective: Construct Validity Failures in LLM Agent Commerce Evaluationcited by 2
Built on by
Citing papers per year: 2026: 1.
1 paper.