# Beyond the Final Answer: Evaluating the Reasoning Trajectories of Tool-Augmented Agents

arXiv: https://arxiv.org/abs/2510.02837 · HTML: https://mlsanity.com/paper/2510.02837/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning.
- Builds on 31 of its 36 references that are in the map.
- First posted on arXiv 3 Oct 2025.

## Placed in

- LLM reasoning / [Reasoning for agents, planning and code](https://mlsanity.com/lr/reasoning-for-agents-planning-and-code/)
- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 30 of 31)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Large Language Models are Zero-Shot Reasoners](https://mlsanity.com/paper/2205.11916/) (2022)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [GAIA: a benchmark for General AI Assistants](https://mlsanity.com/paper/2311.12983/) (2023)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Models](https://mlsanity.com/paper/2503.09567/) (2025)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving](https://mlsanity.com/paper/2309.17452/) (2023)
- [Agent-as-a-Judge: Evaluate Agents with Agents](https://mlsanity.com/paper/2410.10934/) (2024)
- [Magentic-One: A Generalist Multi-Agent System for Solving Complex Tasks](https://mlsanity.com/paper/2411.04468/) (2024)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://mlsanity.com/paper/2402.05044/) (2024)
- [MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use](https://mlsanity.com/paper/2310.03128/) (2023)
- [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.00212/) (2025)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction](https://mlsanity.com/paper/2401.06201/) (2024)
- [A Multimodal Foundation Agent for Financial Trading: Tool-Augmented, Diversified, and Generalist](https://mlsanity.com/paper/2402.18485/) (2024)
- [Deductive Verification of Chain-of-Thought Reasoning](https://mlsanity.com/paper/2306.03872/) (2023)
- [MMedAgent: Learning to Use Medical Tools with Multi-modal Agent](https://mlsanity.com/paper/2407.02483/) (2024)
- [GTA: A Benchmark for General Tool Agents](https://mlsanity.com/paper/2407.08713/) (2024)
- [MLLM-Tool: A Multimodal Large Language Model For Tool Agent Learning](https://mlsanity.com/paper/2401.10727/) (2024)
- [No Free Labels: Limitations of LLM-as-a-Judge Without Human Grounding](https://mlsanity.com/paper/2503.05061/) (2025)
- [Multi-modal Agent Tuning: Building a VLM-Driven Agent for Efficient Tool Usage](https://mlsanity.com/paper/2412.15606/) (2024)
- [From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions](https://mlsanity.com/paper/2410.08197/) (2024)
- [m&m's: A Benchmark to Evaluate Tool-Use for multi-step multi-modal Tasks](https://mlsanity.com/paper/2403.11085/) (2024)
- [Needle in the Haystack for Memory Based Large Language Models](https://mlsanity.com/paper/2407.01437/) (2024)
- [ToolBeHonest: A Multi-level Hallucination Diagnostic Benchmark for Tool-Augmented Large Language Models](https://mlsanity.com/paper/2406.20015/) (2024)
- [Is Safety Standard Same for Everyone? User-Specific Safety Evaluation of Large Language Models](https://mlsanity.com/paper/2502.15086/) (2025)

## Built on by (most cited first, 24 of 24)

- [Are Tools All We Need? Unveiling the Tool-Use Tax in LLM Agents](https://mlsanity.com/paper/2605.00136/) (2026)
- [Counterfactual Trace Auditing of LLM Agent Skills](https://mlsanity.com/paper/2605.11946/) (2026)
- [Where Do Deep-Research Agents Go Wrong? Span-Level Error Localization in Agent Trajectories](https://mlsanity.com/paper/2606.02060/) (2026)
- [SkillJuror: Measuring How Agent Skill Organization Changes Runtime Behavior](https://mlsanity.com/paper/2606.11543/) (2026)
- [Why Your Deep Research Agent Fails? On Hallucination Evaluation in Full Research Trajectory](https://mlsanity.com/paper/2601.22984/) (2026)
- [When Is Enough Not Enough? Illusory Completion in Search Agents](https://mlsanity.com/paper/2602.07549/) (2026)
- [PAIR: Prefix-Aware Internal Reward Model for Multi-Turn Agent Optimization](https://mlsanity.com/paper/2605.17877/) (2026)
- [Tangent: An Empirical Study of Testing Practices for LLM-Based Agent Applications](https://mlsanity.com/paper/2608.08413/) (2026)
- [trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories](https://mlsanity.com/paper/2609.00038/) (2026)
- [From State to Action: OODA-Tool for Reliable Multi-Turn Tool Use](https://mlsanity.com/paper/2608.24368/) (2026)
- [Waking Up Blind: Cold-Start Optimization of Supervision-Free Agentic Trajectories for Grounded Visual Perception](https://mlsanity.com/paper/2604.17475/) (2026)
- [An Empirical Study of Automating Agent Evaluation](https://mlsanity.com/paper/2605.11378/) (2026)
- [Projecting the Emerging Mindset of SWE Agent by Launching a Wild Code Understanding Journey](https://mlsanity.com/paper/2606.08500/) (2026)
- [VeriGraph: Towards Verifiable Data-Analytic Agents](https://mlsanity.com/paper/2606.16603/) (2026)
- [DynamicMCPBench: A Trace-Grounded, Effect-Scored Benchmark for LLM Agents over Live MCP Servers](https://mlsanity.com/paper/2607.20531/) (2026)
- [Socially Grounded Agentic AI: Coordinating Plural Perspectives through Social Theory](https://mlsanity.com/paper/2608.03910/) (2026)
- [Towards Risk-free AI Agent Deployment](https://mlsanity.com/paper/2608.16411/) (2026)
- [SearchAtlas: Analyzing Agentic Search Strategies via Evidential Query Graphs](https://mlsanity.com/paper/2609.10901/) (2026)
- [Analyzing and Mitigating Cost-Inefficient Behaviors in Coding Agents](https://mlsanity.com/paper/2609.30725/) (2026)
- [When Tools Silently Lie: Evaluating and Mitigating Blind Compliance in Tool-Augmented Data Agents](https://mlsanity.com/paper/2609.37153/) (2026)
- [Lightweight, Rubric-Guided Trajectory Evaluation for Production AI Agents](https://mlsanity.com/paper/2610.03315/) (2026)
- [LongHarness: Advancing Agentic Reasoning over 100M-Token Corpora via Stateful Evidence Management](https://mlsanity.com/paper/2610.13075/) (2026)
- [ACED-Bench: Evaluating How LLM Agents Acquire and Act on Causal Evidence](https://mlsanity.com/paper/2610.14381/) (2026)
- [REPORT-OPS: Hybrid Evidence-Guided Reporting with Human-Supervised Refinement for HPC](https://mlsanity.com/paper/2610.19222/) (2026)

## Abstract (arXiv)

Although recent tool-augmented benchmarks involve complex requests, evaluation remains limited to answer matching, neglecting critical trajectory aspects like efficiency, hallucination, and adaptivity. The most straightforward method for evaluation is to compare an agent's trajectory with the ground-truth, but annotating all valid ground-truth trajectories is prohibitively expensive. In this manner, we introduce TRACE, a reference-free framework for the multi-dimensional evaluation of tool-augmented LLMs. By incorporating an evidence bank which accumulates knowledge from preceding steps, TRACE assesses an agent's reasoning trajectory effectively. To validate our framework, we develop a new meta-evaluation dataset with diverse and flawed trajectories, each labeled with multi-faceted performance scores. Our results confirm that TRACE accurately evaluates complex trajectories even with small open-source LLMs. Furthermore, we apply our method to evaluate the trajectories that agents produce while solving tool-augmented tasks, presenting previously unreported observations and their corresponding insights.