# Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agents

arXiv: https://arxiv.org/abs/2604.16706 · HTML: https://mlsanity.com/paper/2604.16706/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 21 of its 34 references that are in the map.
- First posted on arXiv 17 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 21 of 21)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [Gorilla: Large Language Model Connected with Massive APIs](https://mlsanity.com/paper/2305.15334/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Identifying the Risks of LM Agents with an LM-Emulated Sandbox](https://mlsanity.com/paper/2309.15817/) (2023)
- [State of What Art? A Call for Multi-Prompt LLM Evaluation](https://mlsanity.com/paper/2401.00595/) (2023)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents](https://mlsanity.com/paper/2503.18666/) (2025)
- [Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering](https://mlsanity.com/paper/2307.16877/) (2023)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [Large language models can accurately predict searcher preferences](https://mlsanity.com/paper/2309.10621/) (2023)
- [Perspectives on Large Language Models for Relevance Judgment](https://mlsanity.com/paper/2304.09161/) (2023)
- [Do Language Models Enjoy Their Own Stories? Prompting Large Language Models for Automatic Story Evaluation](https://mlsanity.com/paper/2405.13769/) (2024)
- [A Closer Look at Classification Evaluation Metrics and a Critical Reflection of Common Evaluation Practice](https://mlsanity.com/paper/2404.16958/) (2024)

## Built on by (most cited first, 10 of 10)

- [When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agents](https://mlsanity.com/paper/2606.05806/) (2026)
- [Ratchet: How Reliable Must an LLM Judge Be to Retire a Skill?](https://mlsanity.com/paper/2605.22148/) (2026)
- [AgentCheck: A Reproduce-Intervene-Mitigate Workbench for LLM Agents over MCP](https://mlsanity.com/paper/2607.11098/) (2026)
- [Calibration is the Bottleneck: An Action-Class Diagnostic of Multi-Turn Tool-Calling](https://mlsanity.com/paper/2609.00949/) (2026)
- [Do Agent Benchmarks Do What They Say? An Executable-Contract Audit of Tool-Using Agent Environments](https://mlsanity.com/paper/2609.37315/) (2026)
- [Measurement Without Validity: The Compounding Reliability Problem in Agentic AI Evaluation](https://mlsanity.com/paper/2608.00794/) (2026)
- [World Model Science: Self-Organized Criticality, Weak Chaos, and Metastable Belief Dynamics in Long-Horizon LLM Agents](https://mlsanity.com/paper/2609.17419/) (2026)
- [Hearsay: Can an Auditor Trust the Record a Deployed Agent Harness Writes?](https://mlsanity.com/paper/2609.32495/) (2026)
- [When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents](https://mlsanity.com/paper/2609.37153/) (2026)
- [RSure-Agent: Reliable Use of Tool Observations for Remote Sensing Agents](https://mlsanity.com/paper/2610.04836/) (2026)

## Abstract (arXiv)

Automated evaluation of tool-using large language model (LLM) agents is widely assumed to be reliable, yet this assumption is rarely validated against human annotation. We present AgentProp-Bench, a diagnostic benchmark of 14,750 execution traces from thirteen LLM agents (nine proprietary, four open-weight) across four domains, and use it to audit three questions. First, substring-heuristic judging of agent outputs agrees with human annotation only at chance level (Cohen's kappa = 0.049 against each of two annotators), while a three-LLM ensemble reaches moderate agreement (kappa = 0.432) and a single GPT-4o-mini judge is in fact the strongest (kappa = 0.567); dual-annotator agreement is almost perfect (kappa = 0.835). Second, under validated judging a parameter-level error propagates to a wrong final answer with human-calibrated probability approximately 0.62, replicated across proprietary and open-weight models, and a model's ability to reject corrupted inputs and to recover from them are statistically indepen1). Third, severalagents fabricate tool executions -- asserting tool-derived renever obtained (up toilure invisible toend-to-end scores, and a lightweight runtime interceptor reduhallucination on everto 24 percentagepoints) at a tunable operating point that is net-positive on open-weight models. Ini-2.0-Flash, whichrarely emits an injectable tool call. All code, data, traces, and human labels are relehttps://github.com/bhaskargurram-ai/agenthallu-bench