# Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systems

arXiv: https://arxiv.org/abs/2512.12791 · HTML: https://mlsanity.com/paper/2512.12791/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 14 Dec 2025.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 20 of 20)

- [A Survey on LLM-as-a-Judge](https://mlsanity.com/paper/2411.15594/) (2024)
- [Evaluating Very Long-Term Conversational Memory of LLM Agents](https://mlsanity.com/paper/2402.17753/) (2024)
- [A Survey on the Memory Mechanism of Large Language Model based Agents](https://mlsanity.com/paper/2404.13501/) (2024)
- [Why Do Multi-Agent LLM Systems Fail?](https://mlsanity.com/paper/2503.13657/) (2025)
- [AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents](https://mlsanity.com/paper/2410.09024/) (2024)
- [Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions](https://mlsanity.com/paper/2507.05257/) (2025)
- [Agent-as-a-Judge: Evaluate Agents with Agents](https://mlsanity.com/paper/2410.10934/) (2024)
- [Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systems](https://mlsanity.com/paper/2505.00212/) (2025)
- [AGENTIF: Benchmarking Instruction Following of Large Language Models in Agentic Scenarios](https://mlsanity.com/paper/2505.16944/) (2025)
- [AgentOps: Enabling Observability of LLM Agents](https://mlsanity.com/paper/2411.05285/) (2024)
- [Agentic Software Engineering: Foundational Pillars and a Research Roadmap](https://mlsanity.com/paper/2509.06216/) (2025)
- [AIOpsLab: A Holistic Framework to Evaluate AI Agents for Enabling Autonomous Clouds](https://mlsanity.com/paper/2501.06706/) (2025)
- [MCPToolBench++: A Large Scale AI Agent Model Context Protocol MCP Tool Use Benchmark](https://mlsanity.com/paper/2508.07575/) (2025)
- [Self-Adaptive Large Language Model (LLM)-Based Multiagent Systems](https://mlsanity.com/paper/2307.06187/) (2023)
- [Do LLMs Overcome Shortcut Learning? An Evaluation of Shortcut Challenges in Large Language Models](https://mlsanity.com/paper/2410.13343/) (2024)
- [An Empirical Study of Testing Practices in Open Source AI Agent Frameworks and Agentic Applications](https://mlsanity.com/paper/2509.19185/) (2025)
- [Taming Uncertainty via Automation: Observing, Analyzing, and Optimizing Agentic AI Systems](https://mlsanity.com/paper/2507.11277/) (2025)
- [Improving Google A2A Protocol: Protecting Sensitive Data and Mitigating Unintended Harms in Multi-Agent Systems](https://mlsanity.com/paper/2505.12490/) (2025)
- [Uncertainty in Self-Adaptive Systems: A Research Community Perspective](https://mlsanity.com/paper/2103.02717/) (2021)
- [Engineering LLM Powered Multi-agent Framework for Autonomous CloudOps](https://mlsanity.com/paper/2501.08243/) (2025)

## Built on by (most cited first, 8 of 8)

- [MemoryArena: Benchmarking Agent Memory in Interdependent Multi-Session Agentic Tasks](https://mlsanity.com/paper/2602.16313/) (2026)
- [The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration](https://mlsanity.com/paper/2603.22862/) (2026)
- [Graphectory Viewer: A Tool for Process-Centric Analysis of Agentic Software Trajectories](https://mlsanity.com/paper/2608.17195/) (2026)
- [Iterative Audit Convergence in LLM-Managed Multi-Agent Systems: A Case Study in Prompt-Engineering Quality Assurance](https://mlsanity.com/paper/2605.12280/) (2026)
- [Beyond Component Testing: Validating Agentic AI Systems](https://mlsanity.com/paper/2607.29405/) (2026)
- [Forward-Deployed Full-Stack Engineering for Autonomous Cloud MLOps](https://mlsanity.com/paper/2608.29615/) (2026)
- [Continuous Process-Level Evaluation for Evolving Enterprise AI Agent Skills](https://mlsanity.com/paper/2610.01833/) (2026)
- [AgentStripes: An Evidence-Graded Maturity Model for AI Agents](https://mlsanity.com/paper/2610.18603/) (2026)

## Abstract (arXiv)

Recent advances in agentic AI have shifted the focus from standalone Large Language Models (LLMs) to integrated systems that combine LLMs with tools, memory, and other agents to perform complex tasks. These multi-agent architectures enable coordinated reasoning, planning, and execution across diverse domains, allowing agents to collaboratively automate complex workflows. Despite these advances, evaluation and assessment of LLM agents and the multi-agent systems they constitute remain a fundamental challenge. Although various approaches have been proposed in the software engineering literature for evaluating conventional software components, existing methods for AI-based systems often overlook the non-deterministic nature of models. This non-determinism introduces behavioral uncertainty during execution, yet existing evaluations rely on binary task completion metrics that fail to capture it. Evaluating agentic systems therefore requires examining additional dimensions, including the agent ability to invoke tools, ingest and retrieve memory, collaborate with other agents, and interact effectively with its environment. These challenges emerged during our ongoing industry collaboration with MontyCloud Inc., when we deployed an agentic system in production. These limitations surfaced during deployment, highlighting practical gaps in the current evaluation methods and the need for a systematic assessment of agent behavior beyond task outcomes. Informed by these observations and established definitions of agentic systems, we propose an end-to-end Agent Assessment Framework with four evaluation pillars encompassing LLMs, Memory, Tools, and Environment. We validate the framework on a representative Autonomous CloudOps use case, where experiments reveal behavioral deviations overlooked by conventional metrics, demonstrating its effectiveness in capturing runtime uncertainties.