mlsanityData to 14 Oct 2026

AgentStripes: An Evidence-Graded Maturity Model for AI Agents

arXiv 2610.18603 ↗First posted 13 Oct 2026cs.AI cs.CY cs.SYcited by 0

Not on the map. Why?

Abstract (arXiv)

Benchmarks report what an Artificial Intelligence (AI) agent can do on a task suite, but not whether one released version has been engineered with the controls that make it reliable, safe and observable, or how strongly that is shown. This paper proposes AgentStripes, an evidence-graded maturity model for AI agents. It defines levels (0 to 7 stripes), process areas (12 areas holding 40 capabilities), three gates (basic, safety, proof) that hold an agent at a level however high its score, and evidence grades E0 to E5. Each applicable capability receives a rating from 0 to 3 and a separate evidence level; these are combined into a credit, aggregated into a severity-weighted score and converted into stripes that the gates limit. Instruments gather evidence, language-model judges interpret it under explicit criteria, and the arithmetic is deterministic. The model is compared with standards at topic level without a claim of conformance, and an unexecuted validation protocol is specified. An empirical evaluation (exploratory, non-confirmatory) of 30 public Python agent repositories, with static evidence only, is a demonstration and not a validation: static evidence caps every final stripe at 2, 26 repositories end at one stripe and 4 at two, and this distinction is unreliable, since re-judging changed the final stripe of 4 of 22 repositories (engine, dossier and panel changed), whereas none of 8 panel-only repeats, all at stripe 1, changed. The parameters are expert conventions without outcome based calibration, and the stripes have not been shown to predict reliability or safety.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 57 of its 110 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

57 of 110 references.

References · 57

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  4. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  5. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  6. 2024LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memorycited by 707
  7. 2025BrowseComp: A Simple Yet Challenging Benchmark for Browsing Agentscited by 647
  8. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  9. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  10. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 437
  11. 2024AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Basescited by 429
  12. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
38 more
  1. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  2. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 321
  3. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agentscited by 306
  4. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  5. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  6. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 161
  7. 2024AssistantBench: Can Web Agents Solve Realistic and Time-Consuming Tasks?cited by 158
  8. 2024ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agentscited by 133
  9. 2025Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluationcited by 92
  10. 2024WorkArena++: Towards Compositional Planning and Reasoning-based Common Knowledge Work Taskscited by 90
  11. 2025AgentAuditor: Human-Level Safety and Security Evaluation for LLM Agentscited by 90
  12. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safetycited by 75
  13. 2025ACEBench: Who Wins the Match Point in Tool Usage?cited by 71
  14. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarioscited by 59
  15. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 57
  16. 2024NESTFUL: A Benchmark for Evaluating LLMs on Nested Sequences of API Callscited by 48
  17. 2026AgentLAB: Benchmarking LLM Agents against Long-Horizon Attackscited by 39
  18. 2025MCPEval: Automatic MCP-based Deep Evaluation for AI Agent Modelscited by 36
  19. 2026ReliabilityBench: Evaluating LLM Agent Reliability Under Production-Like Stress Conditionscited by 36
  20. 2025MCPToolBench++: A Large Scale AI Agent Model Context Protocol MCP Tool Use Benchmarkcited by 32
  21. 2025MCP-AgentBench: Evaluating Real-World Language Agent Performance with MCP-Mediated Toolscited by 32
  22. 2026Mem2ActBench: A Benchmark for Evaluating Long-Term Memory Utilization in Task-Oriented Autonomous Agentscited by 32
  23. 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agentscited by 29
  24. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agentscited by 28
  25. 2025UltraHorizon: Benchmarking Agent Capabilities in Ultra Long-Horizon Scenarioscited by 26
  26. 2025TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systemscited by 26
  27. 2025HCAST: Human-Calibrated Autonomy Software Taskscited by 24
  28. 2026OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agentscited by 22
  29. 2025WAREX: Web Agent Reliability Evaluation on Existing Benchmarkscited by 15
  30. 2025BrowserArena: Evaluating LLM Agents on Real-World Web Navigation Taskscited by 14
  31. 2026MemSyco-Bench: Benchmarking Sycophancy in Agent Memorycited by 12
  32. 2026AgenTRIM: Tool Risk Mitigation for Agentic AIcited by 11
  33. 2025AGENTSAFE: A Unified Framework for Ethical Assurance and Governance in Agentic AIcited by 10
  34. 2025CORE: Full-Path Evaluation of LLM Agents Beyond Final Statecited by 9
  35. 2026EMemBench: Interactive Benchmarking of Episodic Memory for VLM Agentscited by 9
  36. 2025Beyond Accuracy: A Multi-Dimensional Framework for Evaluating Enterprise Agentic AI Systemscited by 8
  37. 2025Beyond Task Completion: An Assessment Framework for Evaluating Agentic AI Systemscited by 8
  38. 2026A Trace-Based Assurance Framework for Agentic AI Orchestration: Contracts, Testing, and Governancecited by 8

50 most cited of 57 All 57 (JSON)

Built on by

Not cited yet.