# Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactions

arXiv: https://arxiv.org/abs/2509.18847 · HTML: https://mlsanity.com/paper/2509.18847/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use.
- Builds on 31 of its 73 references that are in the map.
- First posted on arXiv 23 Sep 2025.

## Placed in

- LLM agents and tool use / [Tool learning and function calling](https://mlsanity.com/agents/tool-learning-and-function-calling/)

## Builds on (most cited first, 30 of 31)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [DAPO: An Open-Source LLM Reinforcement Learning System at Scale](https://mlsanity.com/paper/2503.14476/) (2025)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Qwen2.5-Coder Technical Report](https://mlsanity.com/paper/2409.12186/) (2024)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Group Sequence Policy Optimization](https://mlsanity.com/paper/2507.18071/) (2025)
- [API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs](https://mlsanity.com/paper/2304.08244/) (2023)
- [ReTool: Reinforcement Learning for Strategic Tool Use in LLMs](https://mlsanity.com/paper/2504.11536/) (2025)
- [ToolRL: Reward is All Tool Learning Needs](https://mlsanity.com/paper/2504.13958/) (2025)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Tool Learning with Large Language Models: A Survey](https://mlsanity.com/paper/2405.17935/) (2024)
- [ToolACE: Winning the Points of LLM Function Calling](https://mlsanity.com/paper/2409.00920/) (2024)
- [SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models](https://mlsanity.com/paper/2504.11468/) (2025)
- [ToRL: Scaling Tool-Integrated RL](https://mlsanity.com/paper/2503.23383/) (2025)
- [xLAM: A Family of Large Action Models to Empower AI Agent Systems](https://mlsanity.com/paper/2409.03215/) (2024)
- [Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarios](https://mlsanity.com/paper/2401.17167/) (2024)
- [ToolGen: Unified Tool Retrieval and Calling via Generation](https://mlsanity.com/paper/2410.03439/) (2024)
- [LongCat-Flash Technical Report](https://mlsanity.com/paper/2509.01322/) (2025)
- [From Exploration to Mastery: Enabling LLMs to Master Tools via Self-Driven Interactions](https://mlsanity.com/paper/2410.08197/) (2024)
- [Facilitating Multi-turn Function Calling for LLMs via Compositional Instruction Tuning](https://mlsanity.com/paper/2410.12952/) (2024)
- [LLM4EDA: Emerging Progress in Large Language Models for Electronic Design Automation](https://mlsanity.com/paper/2401.12224/) (2023)
- [Large Language Models Can Self-Correct with Key Condition Verification](https://mlsanity.com/paper/2405.14092/) (2024)
- [PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifier](https://mlsanity.com/paper/2506.10406/) (2025)
- [Self-Reflection Makes Large Language Models Safer, Less Biased, and Ideologically Neutral](https://mlsanity.com/paper/2406.10400/) (2024)
- [Boosting LLM Reasoning via Spontaneous Self-Correction](https://mlsanity.com/paper/2506.06923/) (2025)
- [Equipping Language Models with Tool Use Capability for Tabular Data Analysis in Finance](https://mlsanity.com/paper/2401.15328/) (2024)
- [TL-Training: A Task-Feature-Based Framework for Training Large Language Models in Tool Use](https://mlsanity.com/paper/2412.15495/) (2024)
- [Sparse Rewards Can Self-Train Dialogue Agents](https://mlsanity.com/paper/2409.04617/) (2024)

## Built on by (most cited first, 12 of 12)

- [Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions](https://mlsanity.com/paper/2602.22680/) (2026)
- [The Evolution of Tool Use in LLM Agents: From Single-Tool Call to Multi-Tool Orchestration](https://mlsanity.com/paper/2603.22862/) (2026)
- [Robust Tool Use via Fission-GRPO: Learning to Recover from Execution Errors](https://mlsanity.com/paper/2601.15625/) (2026)
- [Beyond Words and Pixels: A Benchmark for Implicit World Knowledge Reasoning in Generative Models](https://mlsanity.com/paper/2511.18271/) (2025)
- [Correct Is Not Enough: Training Reasoning Planners with Executor-Grounded Rewards](https://mlsanity.com/paper/2605.03862/) (2026)
- [SAP: State-Guided Data Synthesis with Argument Provenance for Multi-Turn Tool Use](https://mlsanity.com/paper/2609.06124/) (2026)
- [Failure-Transparent Agents: Benchmarking Post-Failure Reporting in Tool-Using Language Models](https://mlsanity.com/paper/2609.35732/) (2026)
- [Agent Lifecycle Toolkit (ALTK): Reusable Middleware Components for Robust AI Agents](https://mlsanity.com/paper/2603.15473/) (2026)
- [Mint-Agent: Introducing Finance-Native Agentic Foundation Models](https://mlsanity.com/paper/2608.16386/) (2026)
- [Toolcompass: Guiding Tool Trialing, Not Suppressing It](https://mlsanity.com/paper/2609.25678/) (2026)
- [Towards Reliable AI Data Scientists: Data Agents with Workflow Harnesses](https://mlsanity.com/paper/2609.35255/) (2026)
- [MCP Error Messages Written for Developers Hurt the Most Capable Agents Most](https://mlsanity.com/paper/2609.35381/) (2026)

## Abstract (arXiv)

Tool-augmented large language models (LLMs) are usually trained with supervised imitation or coarse-grained reinforcement learning that optimizes single tool calls. Current self-reflection practices rely on heuristic prompts or one-way reasoning: the model is urged to 'think more' instead of learning error diagnosis and repair. This is fragile in multi-turn interactions; after a failure the model often repeats the same mistake. We propose structured reflection, which turns the path from error to repair into an explicit, controllable, and trainable action. The agent produces a short yet precise reflection: it diagnoses the failure using evidence from the previous step and then proposes a correct, executable follow-up call. For training we combine DAPO and GSPO objectives with a reward scheme tailored to tool use, optimizing the stepwise strategy Reflect, then Call, then Final. To evaluate, we introduce Tool-Reflection-Bench, a lightweight benchmark that programmatically checks structural validity, executability, parameter correctness, and result consistency. Tasks are built as mini trajectories of erroneous call, reflection, and corrected call, with disjoint train and test splits. Experiments on BFCL v3 and Tool-Reflection-Bench show large gains in multi-turn tool-call success and error recovery, and a reduction of redundant calls. These results indicate that making reflection explicit and optimizing it directly improves the reliability of tool interaction and offers a reproducible path for agents to learn from failure.