# A Trust Layer for Agent Evaluation

arXiv: https://arxiv.org/abs/2610.07274 · HTML: https://mlsanity.com/paper/2610.07274/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 24 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 19 of 19)

- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [AI Agents That Matter](https://mlsanity.com/paper/2407.01502/) (2024)
- [The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systems](https://mlsanity.com/paper/2503.03750/) (2025)
- [ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditions](https://mlsanity.com/paper/2601.06112/) (2026)
- [Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use](https://mlsanity.com/paper/2605.02964/) (2026)
- [Agents' Last Exam](https://mlsanity.com/paper/2606.05405/) (2026)
- [EvilGenie: A Reward Hacking Benchmark](https://mlsanity.com/paper/2511.21654/) (2025)
- [BeHonest: Benchmarking Honesty in Large Language Models](https://mlsanity.com/paper/2406.13261/) (2024)
- [Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation](https://mlsanity.com/paper/2603.03116/) (2026)
- [On Randomness in Agentic Evals](https://mlsanity.com/paper/2602.07150/) (2026)
- [BenchGuard: Who Guards the Benchmarks? Automated Auditing of LLM Agent Benchmarks](https://mlsanity.com/paper/2604.24955/) (2026)
- [From Confident Closing to Silent Failure: Characterizing False Success in LLM Agents](https://mlsanity.com/paper/2606.09863/) (2026)
- [Stochasticity in Agentic Evaluations: Quantifying Inconsistency with Intraclass Correlation](https://mlsanity.com/paper/2512.06710/) (2025)
- [On the Reliability of Computer Use Agents](https://mlsanity.com/paper/2604.17849/) (2026)
- [How Much Can Repeated Running Buy? A Measured Variance Ceiling for LLM Code Evaluation](https://mlsanity.com/paper/2606.00920/) (2026)
- [SWE-Doctor: Guiding Software Engineering Agents with Runtime Diagnosis from Multi-Faceted Bug Reproduction Tests](https://mlsanity.com/paper/2607.00990/) (2026)
- [BuildBench: Benchmarking LLM Agents on Compiling Real-World Open-Source Software](https://mlsanity.com/paper/2509.25248/) (2025)
- [GVD: Governed Versioning and Deduplication for Document Repositories](https://mlsanity.com/paper/2609.17696/) (2026)
- [SAGE: Governed Artifact Generation from Enterprise Guidelines](https://mlsanity.com/paper/2609.17775/) (2026)

## Abstract (arXiv)

Deterministic benchmark scores show that an agent received credit, but not whether that credit was earned, reported honestly, or would hold on a second run. We introduce a Trust Layer for Agent Evaluation, an additive post-hoc framework that reports, beside each recorded score, whether it should be believed. It verifies four properties: whether the result is supported by the benchmark's own grading logic, whether a passing answer was earned through traceable computation, whether the agent's completion claim matches what occurred, and whether the result is stable under repeated execution. The first three use only saved artifacts; the fourth re-runs the agent. Model judgments only label evidence under majority voting; all verdicts follow deterministic rules and never modify the recorded score. Applied to five agent configurations on 108 tasks from Agents' Last Exam, every model shows passing runs with no traceable computation (at rates varying tenfold), confirmed false completion claims, and unstable results: 18-46% of tasks do not stay in one score band over five runs. Only 22.6% of recorded passes clear all four checks (95% CI 15.0-32.6, n=84). Measuring what an agent can do and verifying that it did it are different problems, and current benchmarks address only the first.