mlsanityData to 14 Oct 2026

Beyond Accuracy: A Cognitive Load Framework for Mapping the Capability Boundaries of Tool-use Agents

arXiv 2601.20412 ↗First posted 28 Jan 2026cs.CL cs.SEcited by 1

On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks

Abstract (arXiv)

The ability of Large Language Models (LLMs) to use external tools unlocks powerful real-world interactions, making rigorous evaluation essential. However, current benchmarks primarily report final accuracy, revealing what models can do but obscuring the cognitive bottlenecks that define their true capability boundaries. To move from simple performance scoring to a diagnostic tool, we introduce a framework grounded in Cognitive Load Theory. Our framework deconstructs task complexity into two quantifiable components: Intrinsic Load, the inherent structural complexity of the solution path, formalized with a novel Tool Interaction Graph; and Extraneous Load, the difficulty arising from ambiguous task presentation. To enable controlled experiments, we construct ToolLoad-Bench, the first benchmark with parametrically adjustable cognitive load. Our evaluation reveals distinct performance cliffs as cognitive load increases, allowing us to precisely map each model's capability boundary. We validate that our framework's predictions are highly calibrated with empirical results, establishing a principled methodology for understanding an agent's limits and a practical foundation for building more efficient systems.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 25 of its 40 references that are in the map. First posted on arXiv 28 Jan 2026.

Builds on

25 of 40 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025TaskCraft: Automated Generation of Agentic Tasks11 Jun
  2. 2025APIGen-MT: Agentic Pipeline for Multi-Turn Data Generation via Simulated Agent-Human Interplay4 Apr
  3. 2024Small LLMs Are Weak Tool Learners: A Multi-LLM Agent14 Jan
  4. 2023A Survey on Large Language Model based Autonomous Agents22 Aug

Other references · 21

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  7. 2023Augmented Language Models: a Surveycited by 403
  8. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 393
  9. 2025ReTool: Reinforcement Learning for Strategic Tool Use in LLMscited by 392
  10. 2025ToolRL: Reward is All Tool Learning Needscited by 362
  11. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
  12. 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
9 more
  1. 2023T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Stepcited by 93
  2. 2024Tool Learning in the Wild: Empowering Language Models as Automatic Tool Agentscited by 56
  3. 2024Hammer: Robust Function-Calling for On-Device Language Models via Function Maskingcited by 55
  4. 2024Learning to Use Tools via Cooperative and Interactive Agentscited by 54
  5. 2023ToolTalk: Evaluating Tool-Usage in a Conversational Settingcited by 45
  6. 2024AgentStudio: A Toolkit for Building General Virtual Agentscited by 45
  7. 2025Magnet: Multi-turn Tool-use Data Synthesis and Distillation via Graph Translationcited by 37
  8. 2025Sparse Brains are Also Adaptive Brains: Cognitive-Load-Aware Dynamic Activation for LLMscited by 3
  9. 2025Streaming, Fast and Slow: Cognitive Load-Aware Streaming for Efficient LLM Servingcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2025United Minds or Isolated Agents? Exploring Coordination of LLMs under Cognitive Load Theory
  2. 2025Towards a Science of Scaling Agent Systems
  3. 2026From Cognitive Architectures to Language Agents: A Mechanism-Level Review of Lineage, Convergence, and Migration Gaps
  4. 2026Beyond the Leaderboard: A Synthesis of Tool-Use, Planning, and Reasoning Failures in Large Language Model Agents
  5. 2025TRAJECT-Bench:A Trajectory-Aware Benchmark for Evaluating Agentic Tool Use
  6. 2026Agentic Cognitive Depth: Operational Criteria for Evaluating LLM Agents
  7. 2026Capable but Unreliable: Canonical Path Deviation as a Causal Mechanism of Agent Failure in Long-Horizon Tasks
  8. 2026ToolAnchor: Anchoring Counterfactual Context to Boost Agentic Tool-use Capability