mlsanityData to 14 Oct 2026

Exploring Autonomous Agents: A Closer Look at Why They Fail When Completing Tasks

arXiv 2508.13143 ↗First posted 18 Aug 2025cs.AI cs.SEcited by 20

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

Autonomous agent systems powered by Large Language Models (LLMs) have demonstrated promising capabilities in automating complex tasks. However, current evaluations largely rely on success rates without systematically analyzing the interactions, communication mechanisms, and failure causes within these systems. To bridge this gap, we present a benchmark of 34 representative programmable tasks designed to rigorously assess autonomous agents. Using this benchmark, we evaluate three popular open-source agent frameworks combined with two LLM backbones, observing a task completion rate of approximately 50%. Through in-depth failure analysis, we develop a three-tier taxonomy of failure causes aligned with task phases, highlighting planning errors, task execution issues, and incorrect response generation. Based on these insights, we propose actionable improvements to enhance agent planning and self-diagnosis capabilities. Our failure taxonomy, together with mitigation advice, provides an empirical foundation for developing more robust and effective autonomous agent systems in the future.

Key facts. Cited by 20 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 28 of its 32 references that are in the map. First posted on arXiv 18 Aug 2025.

Builds on

28 of 32 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Exploration28 Aug
  2. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
  3. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug

Other references · 25

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversationcited by 1,988
  3. 2023Reasoning with Language Model is Planning with World Modelcited by 861
  4. 2024Executable Code Actions Elicit Better LLM Agentscited by 564
  5. 2024CodeAgent: Enhancing Code Generation with Tool-Integrated Agent Systems for Real-World Repo-level Coding Challengescited by 282
  6. 2023Self-collaboration Code Generation via ChatGPTcited by 258
  7. 2024Large Language Model-Based Agents for Software Engineering: A Surveycited by 178
  8. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  9. 2024LLM-Based Multi-Agent Systems for Software Engineering: Literature Review, Vision and the Road Aheadcited by 155
  10. 2024InfiAgent-DABench: Evaluating Agents on Data Analysis Taskscited by 140
  11. 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agentscited by 133
  12. 2023Keep the Conversation Going: Fixing 162 out of 337 bugs for 0.42 each using ChatGPTcited by 125
13 more
  1. 2023Self-planning Code Generation with Large Language Modelscited by 124
  2. 2023Self-Edit: Fault-Aware Code Editor for Code Generationcited by 111
  3. 2023ChatUniTest: A Framework for LLM-Based Test Generationcited by 96
  4. 2023TaskWeaver: A Code-First Agent Frameworkcited by 86
  5. 2024MarsCode Agent: AI-native Automated Bug Fixingcited by 71
  6. 2024SUPER: Evaluating Agents on Setting Up and Executing Tasks from Research Repositoriescited by 52
  7. 2024CYCLE: Learning to Self-Refine the Code Generationcited by 42
  8. 2023Make LLM a Testing Expert: Bringing Human-like Interaction to Mobile GUI Testing via Functionality-aware Decisionscited by 41
  9. 2024MARE: Multi-Agents Collaboration Framework for Requirements Engineeringcited by 28
  10. 2024PerfCodeGen: Improving Performance of LLM Generated Code with Execution Feedbackcited by 24
  11. 2024Requirements are All You Need: From Requirements to Code with LLMscited by 22
  12. 2025Multi-stage Large Language Model Pipelines Can Outperform GPT-4o in Relevance Assessmentcited by 7
  13. 2024CIBench: Evaluating Your LLMs with a Code Interpreter Plugincited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 19.

20 papers.

Sort:

By area:

  1. 2026ARGUS: Defending LLM Agents Against Context-Aware Prompt Injectioncited by 14
  2. 2026Inference-Time Budget Control for LLM Search Agentscited by 13
  3. 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectoriescited by 13
  4. 2026Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineeringcited by 12
  5. 2026Characterizing Faults in Agentic AI: A Taxonomy of Types, Symptoms, and Root Causescited by 12
  6. 2026Spend Less, Reason Better: Budget-Aware Value Tree Search for LLM Agentscited by 11
  7. 2026TrajAudit: Automated Failure Diagnosis for Agentic Coding Systemscited by 11
  8. 2025Demystifying the Lifecycle of Failures in Platform-Orchestrated Agentic Workflowscited by 10
  9. 2026From Fluent to Verifiable: Claim-Level Auditability for Deep Research Agentscited by 7
  10. 2026Efficient Failure Management for Multi-Agent Systems with Reasoning Trace Representationcited by 6
  11. 2026Beyond Individual Intelligence: Surveying Collaboration, Failure Attribution, and Self-Evolution in LLM-based Multi-Agent Systemscited by 6
  12. 2026When Planning Fails Despite Correct Execution: On Epistemic Calibration for LLM-Based Multi-Agent Systemscited by 3
8 more
  1. 2026Engineering Reliable Coding Agents: Evaluating and Operating the System Around the Modelcited by 3
  2. 2026Profile-Then-Reason: Bounded Semantic Complexity for Tool-Augmented Language Agentscited by 2
  3. 2026Understanding Bugs in Modern Agentic Frameworks: A Study of Symptoms, Root Causes, and Triggering Conditionscited by 2
  4. 2026Beyond the Attention Stability Boundary: Agentic Self-Synthesizing Reasoning Protocolscited by 2
  5. 2026When Agents Look the Same: Quantifying Distillation-Induced Similarity in Tool-Use Behaviorscited by 1
  6. 2026SkillTV-Bench: Benchmarking How Well Judges Perform on Skill-Augmented Agentic Executioncited by 1
  7. 2026Diagnosis-Driven Automatic Repair for Agentic Workflow via Symbolic Inferencecited by 0
  8. 2026A Comprehensive Study of Implementation Bugs in Multi-modal Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025How Do LLMs Fail In Agentic Scenarios? A Qualitative Analysis of Success and Failure Scenarios of Various LLMs in Agentic Simulations
  2. 2026AgentCompass: A Unified Evaluation Infrastructure for Agent Capabilities
  3. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  4. 2025Where LLM Agents Fail and How They can Learn From Failures
  5. 2026A Self-Healing Framework for Reliable LLM-Based Autonomous Agents
  6. 2024AgentQuest: A Modular Benchmark Framework to Measure Progress and Improve LLM Agents
  7. 2025Aegis: Taxonomy and Optimizations for Overcoming Agent-Environment Failures in LLM Agents
  8. 2026ORAgentBench: Can LLM Agents Solve Challenging Operations Research Tasks End to End?