Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment
On the map: LLM agents and tool use → Agent threats, guardrails and governance
also in 1 more
Abstract (arXiv)
Coding agents powered by large language models are becoming central modules of modern IDEs, helping users perform complex tasks by invoking tools. While powerful, tool invocation opens a substantial attack surface. Prior work has demonstrated attacks against general-purpose and domain-specific agents, but none have focused on the security risks of tool invocation in coding agents. To fill this gap, we conduct the first systematic red-teaming of six popular real-world coding agents: Cursor, Claude Code, Copilot, Windsurf, Cline, and Trae. Our red-teaming proceeds in two phases. In Phase 1, we perform prompt leakage reconnaissance to recover system prompts. We discover a general vulnerability, ToolLeak, which allows malicious prompt exfiltration through benign argument retrieval during tool invocation. In Phase 2, we hijack the agent's tool-invocation behavior using a novel two-channel prompt injection in the tool description and return values, achieving remote code execution (RCE). We adaptively construct payloads using security information leaked in Phase 1. In emulation across five backends, our method outperforms baselines on Claude-Sonnet-4, Claude-Sonnet-4.5, Grok-4, and GPT-5. On real agents, our approach succeeds on 19 of 25 agent-LLM pairs, achieving leakage on every agent using Claude and Grok backends. For tool-invocation hijacking, we obtain RCE on every tested agent-LLM pair, with our two-channel method delivering the highest success rate. We provide case studies on Cursor and Claude Code, analyze security guardrails of external and built-in tools, and conclude with practical defense recommendations.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 46 of its 91 references that are in the map. First posted on arXiv 6 Sep 2025.
Builds on
46 of 91 references.
Closest ancestors · 14
Same direction, newest first
- 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
- 2025When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents12 Jul
- 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution1 Jun
- 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents9 May
- 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
- 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injection19 Apr
- 2025StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models14 Apr
- 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits2 Apr
- 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
- 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
- 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agents12 Nov
- 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
2 more
Other references · 32
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
- 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
- 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
- 2023Detecting Language Model Attacks with Perplexitycited by 411
- 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
- 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
- 2023Tool Learning with Foundation Modelscited by 302
20 more
- 2024AutoCodeRover: Autonomous Program Improvementcited by 272
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
- 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
- 2024Tool Learning with Large Language Models: A Surveycited by 221
- 2024ToolACE: Winning the Points of LLM Function Callingcited by 204
- 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
- 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
- 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
- 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 103
- 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
- 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
- 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64
- 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 62
- 2025Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategiescited by 42
- 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agentscited by 32
- 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Modelscited by 27
- 2024Prompt Leakage effect and defense strategies for multi-turn LLM interactionscited by 23
- 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
- 2024Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translationcited by 14
- 2025System Prompt Extraction Attacks and Defenses in Large Language Modelscited by 6
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluationcited by 5
- 2026ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCPcited by 1
- 2026A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Developmentcited by 0
- 2026Where Is the Tradeoff in Using Third-Party API Routers for Agentic Software Development?cited by 0
- 2026Where Is the Tradeoff in Using Third-Party API Routers for Agentic Software Development?26 Jul 2026
- 2026ShareLock: A Stealthy Multi-Tool Threshold Poisoning Attack Against MCP25 Jun 2026
- 2026A Security Analysis of Long-Horizon Agentic AI Systems: Threats, Evaluation, and Framework Development12 Jun 2026
- 2026Toward Secure LLM Agents: Threat Surfaces, Attacks, Defenses, and Evaluation9 Jun 2026
Similar, not cited
Close in wording, no citation either way
- 2025RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents
- 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems
- 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
- 2026MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring
- 2025Takedown: How It's Done in Modern Coding Agent Exploits
- 2026PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
- 2026SIR: Self-improving Red-teaming for Compute Use Agents
- 2026Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines