mlsanityData to 14 Oct 2026

Open, Reliable, and Collective: A Community-Driven Framework for Tool-Using AI Agents

arXiv 2604.00137 ↗First posted 31 Mar 2026cs.AI cs.SEcited by 1

On the map: LLM agents and tool use → Tool learning and function calling

Abstract (arXiv)

Tool-integrated LLMs retrieve information, perform computations, and take real-world actions, but their reliability depends on both tool-use accuracy and intrinsic tool accuracy, including tool correctness, stability, and safety. While prior work primarily emphasizes tool use, intrinsic tool accuracy remains underexamined. We introduce OpenTools, a community-driven and maintainable toolbox for discovering, using, evaluating, and contributing open-source tools. OpenTools standardizes tool interfaces, converts documented Python functions into reviewable bundles, supports maintainer-triggered evaluation, and combines non-executing risk inspection with optional advisory LLM review. A public web demo allows users to run tools and agents, inspect evidence, contribute tests, and submit tools for maintainer review, while MCP enables controlled access from external applications. Experiments show that community-contributed, task-specific tools yield relative gains of 6% to 22% over an existing toolbox across multiple agent architectures, highlighting the importance of intrinsic tool accuracy.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Tool learning and function calling, one of 16 directions in LLM agents and tool use. Builds on 27 of its 43 references that are in the map. First posted on arXiv 31 Mar 2026.

Builds on

27 of 43 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Improving Large Language Models Function Calling and Interpretability via Guided-Structured Templates22 Sep
  2. 2025ToolFuzz -- Automated Agent Tool Testing6 Mar
  3. 2024ToolScan: A Benchmark for Characterizing Errors in Tool-Use LLMs20 Nov
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs31 Jul
  5. 2023Gorilla: Large Language Model Connected with Massive APIs24 May
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Tools9 Feb

Other references · 21

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  3. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  4. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  5. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  6. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  9. 2020PathVQA: 30000+ Questions for Medical Visual Question Answeringcited by 475
  10. 2021SLAKE: A Semantically-Labeled Knowledge-Enhanced Dataset for Medical Visual Question Answeringcited by 449
  11. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 268
  12. 2024Chain of Thought Empowers Transformers to Solve Inherently Serial Problemscited by 260
9 more
  1. 2022CLEVR-Math: A Dataset for Compositional Language, Visual and Mathematical Reasoningcited by 120
  2. 2024MMedAgent: Learning to Use Medical Tools with Multi-modal Agentcited by 109
  3. 2024Chain of Preference Optimization: Improving Chain-of-Thought Reasoning in LLMscited by 96
  4. 2023Can Knowledge Graphs Reduce Hallucinations in LLMs? : A Surveycited by 92
  5. 2024PuzzleVQA: Diagnosing Multimodal Reasoning Challenges of Language Models with Abstract Visual Patternscited by 57
  6. 2024SciFIBench: Benchmarking Large Multimodal Models for Scientific Figure Interpretationcited by 48
  7. 2025Attention-guided Self-reflection for Zero-shot Hallucination Detection in Large Language Modelscited by 16
  8. 2025CheMatAgent: Enhancing LLMs for Chemistry and Materials Science through Tree-Search Based Tool Learningcited by 15
  9. 2025Towards autonomous quantum physics research using LLM agents with access to intelligent toolscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Weighted Memory Tree: Remembering What Matters for Long-Horizon LLM Agentscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026OpaqueToolsBench: Learning Nuances of Tool Behavior Through Interaction
  2. 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agents
  3. 2026AgentEscapeBench: Evaluating Out-of-Domain Tool-Grounded Reasoning in LLM Agents
  4. 2026ComplexMCP: Evaluation of LLM Agents in Dynamic, Interdependent, and Large-Scale Tool Sandbox
  5. 2025GRETEL: A Goal-driven Retrieval and Execution-based Trial Framework for LLM Tool Selection Enhancing
  6. 2025OpenAgentSafety: A Comprehensive Framework for Evaluating Real-World AI Agent Safety
  7. 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
  8. 2026ToolMenuBench: Benchmarking Tool-Menu Filtering Strategies for Reliable and Efficient LLM Agents