# Towards a Science of AI Agent Reliability

arXiv: https://arxiv.org/abs/2602.16666 · HTML: https://mlsanity.com/paper/2602.16666/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Feb 2026.

## Built on by (most cited first, 30 of 65)

- [Log analysis is necessary for credible evaluation of AI agents](https://mlsanity.com/paper/2605.08545/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [SafetyDrift: Predicting When AI Agents Cross the Line Before They Actually Do](https://mlsanity.com/paper/2603.27148/) (2026)
- [Hallucinations Undermine Trust; Metacognition is a Way Forward](https://mlsanity.com/paper/2605.01428/) (2026)
- [The Implementation Lottery: Auditing Idea Reliability in Automated Research](https://mlsanity.com/paper/2607.26587/) (2026)
- [Open-World Evaluations for Measuring Frontier AI Capabilities](https://mlsanity.com/paper/2605.20520/) (2026)
- [High-Risk AI Systems and the Problem of Identity in the European AI Act](https://mlsanity.com/paper/2605.23922/) (2026)
- [Life After Benchmark Saturation: A Case Study of CORE-Bench](https://mlsanity.com/paper/2606.26158/) (2026)
- [On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification](https://mlsanity.com/paper/2608.18066/) (2026)
- [Consistency as a Testable Property: Statistical Methods to Evaluate AI Agent Reliability](https://mlsanity.com/paper/2605.10516/) (2026)
- [Designing for Doubt: The Case for Informed Abstention in Autonomous Agents](https://mlsanity.com/paper/2606.02965/) (2026)
- [Plover: Steering GUI Agents through Plan-Centric Interaction](https://mlsanity.com/paper/2607.15193/) (2026)
- [The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior](https://mlsanity.com/paper/2608.11386/) (2026)
- [The Double Measurement Confound in Agent Benchmarks: De-Scaffolding, Ground-Truth Scoring, and Reliability Beyond the Mean](https://mlsanity.com/paper/2609.09218/) (2026)
- [Where Does Exactly-Once Live? Model, Harness, and Tool-Contract Effects on Duplicate Side Effects in LLM Agents](https://mlsanity.com/paper/2609.29095/) (2026)
- [RemoteShield: Enable Robust Multimodal Large Language Models for Earth Observation](https://mlsanity.com/paper/2604.17243/) (2026)
- [Agent Meltdowns: The Road to Hell Is Paved with Helpful Agents](https://mlsanity.com/paper/2605.19149/) (2026)
- [How Much Coordination Gain Is Real? A Paired Noise-Floor Protocol for Multi-Agent LLM Benchmarks](https://mlsanity.com/paper/2606.20695/) (2026)
- [Who Checks the Citations? Benchmarking Legal Hallucination Detection](https://mlsanity.com/paper/2606.21155/) (2026)
- [A Category Theory Account of AI Identity](https://mlsanity.com/paper/2607.00220/) (2026)
- [The "I Don't Know" Filter: Enhancing Agentic Reliability in Function Calling](https://mlsanity.com/paper/2607.04034/) (2026)
- [The Illusion of Robustness: Aggregate Accuracy Hides Prediction Flips under Task-Irrelevant Context](https://mlsanity.com/paper/2607.12963/) (2026)
- [Decision Making Needs Uncertainty Quantification [Lecture Notes]](https://mlsanity.com/paper/2607.14407/) (2026)
- [Generating Expressive and Customizable Evals for Timeseries Data Analysis Agents with AgentFuel](https://mlsanity.com/paper/2603.12483/) (2026)
- [MarketBench: Evaluating AI Agents as Market Participants](https://mlsanity.com/paper/2604.23897/) (2026)
- [Update Opacity: Epistemic Accessibility and Governance Under AI System Change](https://mlsanity.com/paper/2606.00037/) (2026)
- [Monitoring Agentic Systems Before They're Reliable](https://mlsanity.com/paper/2606.02494/) (2026)
- [A Methodology for Auditable Trustworthiness Levels in AI Lifecycle Governance](https://mlsanity.com/paper/2607.16130/) (2026)
- [What if automating AI R&D triggers an intelligence explosion?](https://mlsanity.com/paper/2609.36054/) (2026)
- [Property-Level Reconstructability of Agent Decisions: An Anchor-Level Pilot Across Vendor SDK Adapter Regimes](https://mlsanity.com/paper/2605.12078/) (2026)

## Abstract (arXiv)

AI agents are increasingly deployed to execute important tasks. While rising accuracy scores on standard benchmarks suggest rapid progress, many agents still continue to fail in practice. This discrepancy highlights a fundamental limitation of current evaluations: compressing agent behavior into a single success metric obscures critical operational flaws. Notably, it ignores whether agents behave consistently across runs, withstand perturbations, fail predictably, or have bounded error severity. Grounded in safety-critical engineering, we provide a holistic performance profile by proposing twelve concrete metrics that decompose agent reliability along four key dimensions: consistency, robustness, predictability, and safety. Evaluating 15 models across two complementary benchmarks, we find that recent capability gains have only yielded small improvements in reliability. By exposing these persistent limitations, our metrics complement traditional evaluations while offering tools for reasoning about how agents perform, degrade, and fail.