How Your Credentials Are Leaked by LLM Agent Skills: An Empirical Study
On the map: LLM agents and tool use → Agent threats, guardrails and governance
Abstract (arXiv)
Large Language Model (LLM) agents increasingly rely on third-party skills that operate within privileged execution environments and routinely handle sensitive credentials, yet how these credentials are leaked remains largely unexplored. To fill this gap, we present the first large-scale empirical study on credential leakage in agent skills. From 170,226 artifacts on SkillsMP, the largest open-source skill marketplace, we sampled 17,022 skills via stratified random sampling and analyzed each through static secret extraction (regex and AST parsing), dynamic sandbox testing with mock credentials, and cross-referencing developer intent against runtime behavior. Our analysis identifies 520 affected skills containing 1,708 security issues, and yields a taxonomy of 10 leakage patterns. Three findings stand out. First, 76.3% of cases require jointly analyzing natural-language descriptions and programming logic, showing that credential exposure in skills is fundamentally cross-modal. Second, debug logging accounts for 73.5% of vulnerabilities because agent frameworks feed stdout into the LLM context window, turning routine debugging into a credential exposure vector. Third, 89.6% of leaked credentials are immediately exploitable -- 92.5% during routine execution without elevated privileges -- and the fork-based distribution model defeats remediation, as secrets removed from 107 upstream repositories persist across 50+ independent forks. Following responsible disclosure, all malicious skills have been removed and 91.6% of hardcoded cases remediated. We release our dataset, taxonomy, and detection pipeline to support future agent security research.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 16 of its 57 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
16 of 57 references.
Closest ancestors · 11
Same direction, newest first
- 2026AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations11 Mar
- 2026Formal Analysis and Supply Chain Security for Agentic AI Skills27 Feb
- 2026Agents of Chaos23 Feb
- 2026MalTool: Malicious Tool Attacks on LLM Agents12 Feb
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems24 Jan
- 2026Agent Skills in the Wild: An Empirical Study of Security Vulnerabilities at Scale15 Jan
- 2025Agent Skills Enable a New Class of Realistic and Trivially Simple Prompt Injections30 Oct
- 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
- 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits2 Apr
- 2024Imprompter: Tricking LLM Agents into Improper Tool Use19 Oct
- 2024IsolateGPT: An Execution Isolation Architecture for LLM-Based Agentic Systems8 Mar
Other references · 5
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2019Small World with High Risks: A Study of Security Threats in the npm Ecosystemcited by 36
- 2021What are Weak Links in the npm Supply Chain?cited by 16
- 2024SoK: Analysis of Software Supply Chain Security by Establishing Secure Design Propertiescited by 5
- 2024GoSurf: Identifying Software Supply Chain Attack Vectors in Gocited by 1
Built on by
Citing papers per year: 2026: 14.
14 papers.
Sort:
By area:
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 14
- 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skillscited by 9
- 2026RouteGuard: Internal-Signal Detection of Skill Poisoning in LLM Agentscited by 6
- 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verificationcited by 6
- 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malwarecited by 6
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
- 2026Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skillscited by 4
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systemscited by 2
- 2026SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agentscited by 1
- 2026Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systemscited by 1
- 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agentscited by 0
- 2026OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agentscited by 0
- 2026Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems30 Sep 2026
- 2026"Nothing to See Here'': Unintended Disclosure through Revision Traces of LLM Deliverables28 Sep 2026
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection20 Aug 2026
- 2026OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agents8 Aug 2026
- 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agents28 Jul 2026
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries11 Jul 2026
- 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware2 Jul 2026
- 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification2 Jul 2026
- 2026SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents1 Jun 2026
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems30 May 2026
- 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills13 May 2026
- 2026Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skills10 May 2026
2 more
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystemscited by 14
- 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skillscited by 9
- 2026RouteGuard: Internal-Signal Detection of Skill Poisoning in LLM Agentscited by 6
- 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verificationcited by 6
- 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malwarecited by 6
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Librariescited by 6
- 2026Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skillscited by 4
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systemscited by 2
- 2026SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agentscited by 1
- 2026Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systemscited by 1
- 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agentscited by 0
- 2026OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agentscited by 0
12 most cited of 13 All 13 (JSON)
- 2026Covert Assistance: Helpful LLM Agents Evade Oversight in Multi-Agent Systems30 Sep 2026
- 2026MaliciousSkillBench: A Comprehensive Benchmark for Malicious Agent Skill Detection20 Aug 2026
- 2026OBLIVION: Workflow-Level Operational Skill Unlearning for Deployed Agents8 Aug 2026
- 2026StealthBench: Measuring Operational Stealth in Autonomous Offensive-Security Agents28 Jul 2026
- 2026Dynamic Agent Skills: A Lifecycle Survey and Taxonomy of Evolving Skill Libraries11 Jul 2026
- 2026Cloak and Detonate: Scanner Evasion and Dynamic Detection of Agent Skill Malware2 Jul 2026
- 2026Safety Testing LLM Agents at Scale: From Risk Discovery to Evidence-Grounded Verification2 Jul 2026
- 2026SeClaw: Spec-Driven Security Task Synthesis for Evaluating Autonomous Agents1 Jun 2026
- 2026When Safe Skills Collide: Measuring Compositional Risk in Agent Skill Ecosystems30 May 2026
- 2026AgentTrap: Measuring Runtime Trust Failures in Third-Party Agent Skills13 May 2026
- 2026Trust Me, Import This: Dependency Steering Attacks via Malicious Agent Skills10 May 2026
- 2026SUDP: Secret-Use Delegation Protocol for Agentic Systems27 Apr 2026
12 newest of 13 All 13 (JSON)
Similar, not cited
Close in wording, no citation either way
- 2026"Do Not Mention This to the User": Detecting and Understanding Malicious Agent Skills in the Wild
- 2026Skill-Inject: Measuring Agent Vulnerability to Skill File Attacks
- 2026Black-Box Skill Stealing Attack from Proprietary LLM Agents: An Empirical Study
- 2026Skill-Use: Can LLMs Actually Use Skills in Agentic Harnesses?
- 2026SkillsMetric: Mapping the Detection Boundary of Static Analysis for Malicious Agent Skills
- 2026You Told Me to Do It: Measuring Instructional Text-induced Private Data Leakage in LLM Agents
- 2026From Anatomy to Smells: An Empirical Study of SKILL.md in Agent Skills
- 2026Local LLM Agents as Vulnerable Runtimes:A Source-Code Audit of the Agent Runtime Layer