mlsanityData to 14 Oct 2026

NesTools: A Dataset for Evaluating Nested Tool Learning Abilities of Large Language Models

arXiv 2410.11805 ↗First posted 15 Oct 2024cs.CLcited by 4

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Large language models (LLMs) combined with tool learning have gained impressive results in real-world applications. During tool learning, LLMs may call multiple tools in nested orders, where the latter tool call may take the former response as its input parameters. However, current research on the nested tool learning capabilities is still under-explored, since the existing benchmarks lack relevant data instances. To address this problem, we introduce NesTools to bridge the current gap in comprehensive nested tool learning evaluations. NesTools comprises a novel automatic data generation method to construct large-scale nested tool calls with different nesting structures. With manual review and refinement, the dataset is in high quality and closely aligned with real-world scenarios. Therefore, NesTools can serve as a new benchmark to evaluate the nested tool learning abilities of LLMs. We conduct extensive experiments on 22 LLMs, and provide in-depth analyses with NesTools, which shows that current LLMs still suffer from the complex nested tool learning task.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 24 of its 30 references that are in the map. First posted on arXiv 15 Oct 2024.

Builds on

24 of 30 references.

Closest ancestors · 10

Same direction, newest first

  1. 2024Tool Learning with Large Language Models: A Survey28 May
  2. 2024Seal-Tools: Self-Instruct Tool Learning Dataset for Agent Tuning and Detailed Benchmark14 May
  3. 2024API-BLEND: A Comprehensive Corpora for Training and Benchmarking API LLMs23 Feb
  4. 2024ToolEyes: Fine-Grained Evaluation for Tool Learning Capabilities of Large Language Models in Real-world Scenarios1 Jan
  5. 2023MetaTool Benchmark for Large Language Models: Deciding Whether to Use Tools and Which to Use4 Oct
  6. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  7. 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Cases8 Jun
  8. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  9. 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMs14 Apr
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 14

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Qwen Technical Reportcited by 3,344
  5. 2023Mistral 7Bcited by 3,164
  6. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  7. 2024Qwen2 Technical Reportcited by 2,120
  8. 2023TaskBench: Benchmarking Large Language Models for Task Automationcited by 144
  9. 2023RestGPT: Connecting Large Language Models with Real-World RESTful APIscited by 135
  10. 2024LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-trainingcited by 123
  11. 2023T-Eval: Evaluating the Tool Utilization Capability of Large Language Models Step by Stepcited by 93
  12. 2024Planning, Creation, Usage: Benchmarking LLMs for Comprehensive Tool Utilization in Real-World Complex Scenarioscited by 86
2 more
  1. 2024SciAgent: Tool-augmented Language Models for Scientific Reasoningcited by 70
  2. 2023MultiTool-CoT: GPT-3 Can Use Multiple External Tools with Chain of Thought Promptingcited by 27

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 2.

4 papers.

  1. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  2. 2025MCP-Flow: Facilitating LLM Agents to Master Real-World, Diverse and Scaling MCP Toolscited by 11
  3. 2025In-N-Out: A Parameter-Level API Graph Dataset for Tool Agentscited by 1
  4. 2026VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policiescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ToLeaP: Rethinking Development of Tool Learning with Large Language Models
  2. 2024WTU-EVAL: A Whether-or-Not Tool Usage Evaluation Benchmark for Large Language Models
  3. 2025ToolHop: A Query-Driven Benchmark for Evaluating Large Language Models in Multi-Hop Tool Use
  4. 2024ToolNet: Connecting Large Language Models with Massive Tools via Tool Graph
  5. 2025CRITICTOOL: Evaluating Self-Critique Capabilities of Large Language Models in Tool-Calling Error Scenarios
  6. 2025FamilyTool: A Multi-hop Personalized Tool Use Benchmark
  7. 2024RoTBench: A Multi-Level Benchmark for Evaluating the Robustness of Large Language Models in Tool Learning
  8. 2025Alignment for Efficient Tool Calling of Large Language Models