AgentStripes: An Evidence-Graded Maturity Model for AI Agents
Not on the map. Why?
Abstract (arXiv)
Benchmarks report what an Artificial Intelligence (AI) agent can do on a task suite, but not whether one released version has been engineered with the controls that make it reliable, safe and observable, or how strongly that is shown. This paper proposes AgentStripes, an evidence-graded maturity model for AI agents. It defines levels (0 to 7 stripes), process areas (12 areas holding 40 capabilities), three gates (basic, safety, proof) that hold an agent at a level however high its score, and evidence grades E0 to E5. Each applicable capability receives a rating from 0 to 3 and a separate evidence level; these are combined into a credit, aggregated into a severity-weighted score and converted into stripes that the gates limit. Instruments gather evidence, language-model judges interpret it under explicit criteria, and the arithmetic is deterministic. The model is compared with standards at topic level without a claim of conformance, and an unexecuted validation protocol is specified. An empirical evaluation (exploratory, non-confirmatory) of 30 public Python agent repositories, with static evidence only, is a demonstration and not a validation: static evidence caps every final stripe at 2, 26 repositories end at one stripe and 4 at two, and this distinction is unreliable, since re-judging changed the final stripe of 4 of 22 repositories (engine, dossier and panel changed), whereas none of 8 panel-only repeats, all at stripe 1, changed. The parameters are expert conventions without outcome based calibration, and the stripes have not been shown to predict reliability or safety.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 110 references that are in the map. First posted on arXiv 13 Oct 2026.
Builds on
57 of 110 references.
References · 57
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2023GAIA: a benchmark for General AI Assistantscited by 1,128
- 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
- 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
- 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
38 more
- 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
- 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
- 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
- 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
- 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
- 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
- 2025Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluationcited by 92
- 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 90
- 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agentscited by 90
- 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 75
- 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 71
- 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 59
- 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 57
- 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Callscited by 48
- 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attackscited by 39
- 2025MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Modelscited by 36
- 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditionscited by 36
- 2025MCPToolBench++: A Large Scale AI Agent Model Context Protocol MCP Tool Use Benchmarkcited by 32
- 2025MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Toolscited by 32
- 2026Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agentscited by 32
- 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agentscited by 29
- 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agentscited by 28
- 2025UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarioscited by 26
- 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systemscited by 26
- 2025HCAST: Human-Calibrated Autonomy Software Taskscited by 24
- 2026OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agentscited by 22
- 2025WAREX: Web Agent Reliability Evaluation on Existing Benchmarkscited by 15
- 2025BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Taskscited by 14
- 2026MemSyco-Bench: Benchmarking Sycophancy in Agent Memorycited by 12
- 2026AgenTRIM: Tool Risk Mitigation for Agentic AIcited by 11
- 2025AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AIcited by 10
- 2025CORE: Full-Path Evaluation of LLM Agents Beyond Final Statecited by 9
- 2026EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agentscited by 9
- 2025Beyond Accuracy: A Multi-Dimensional Framework for Evaluating Enterprise Agentic AI Systemscited by 8
- 2025Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systemscited by 8
- 2026A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governancecited by 8
50 most cited of 57 All 57 (JSON)
Built on by
Not cited yet.