Evaluating Tool Cloning in Agentic-AI Ecosystems
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Agent tools are becoming a core interface through which LLM agents access external data, services, and execution environments. As these tools are distributed through public marketplaces, raw tool counts may substantially overstate ecosystem diversity if many repositories are cloned, lightly modified, or derived from shared templates. Such hidden duplication can contaminate benchmark splits, propagate vulnerable implementations, bias measurements of tool-use generalization, and raise provenance, attribution, and intellectual-property concerns. We present, to our knowledge, the first large-scale measurement study of tool cloning in agentic AI ecosystems. We curate a unified dataset from multiple public platforms, covering 7,508 Model Context Protocol (MCP) repositories with 87,564 extracted tools and 1,353 Skills repositories with 12,447 tools, for a total of 8,861 repositories and 100,011 tool entries. To measure implementation-level duplication, we build a repository-level auditing pipeline using complementary lexical and fuzzy-structural similarity metrics, and compute pairwise similarity across MCP-to-MCP, Skills-to-Skills, and MCP-to-Skills repository pairs. We further manually verify 100 sampled pairs per MCP and Skills ecosystem across similarity-score buckets to calibrate how often high similarity reflects true code cloning. Our analysis shows that cloning is not an isolated artifact: high-similarity regions appear across comparison settings, and 60% of high-Jaccard candidates and 85% of high-ssdeep candidates in the MCP ecosystem are manually verified as clones. These results indicate that tool cloning is a pervasive and severe source of hidden duplication in agent-tool ecosystems. They further suggest that agent-tool datasets and benchmarks should account for repository provenance and implementation similarity when measuring tool diversity or constructing evaluation splits.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 12 of its 30 references that are in the map. First posted on arXiv 10 May 2026.
Builds on
12 of 30 references.
References · 12
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
- 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2023ToolAlpaca: Generalized Tool Learning for Language Models with 3000 Simulated Casescited by 383
- 2015SourcererCC: Scaling Code Clone Detection to Big Codecited by 40
- 2017Do Code Clones Matter?cited by 12
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Govern the Repository, Not the Agent: Measuring Ecosystem-Level Risk in AI-Native Software
- 2026Latent Reuse in Agent Skills: Multi-modal Clone Detection at Ecosystem Scale
- 2026Silent Failures in Agent-Tool Interaction: An Audit of ToolUniverse
- 2026Open, Reliable, and Collective: A Community-Driven Framework for Tool-Using AI Agents
- 2026ToolRosella: Translating Code Repositories into Standardized Tools for Scientific Agents
- 2026AgenticFlict: A Large-Scale Dataset of Merge Conflicts in AI Coding Agent Pull Requests on GitHub
- 2026AgentTrails: Towards Trust and Reuse for Agentic Tasks
- 2025Towards a Standard, Enterprise-Relevant Agentic AI Benchmark: Lessons from 5.5 billion tokens' worth of agentic AI evaluations