mlsanityData to 7 Oct 2026

Red-Teaming Coding Agents from a Tool-Invocation Perspective: An Empirical Security Assessment

arXiv 2509.05755 ↗First posted 6 Sep 2025cs.CR cs.AIcited by 4

On the map: LLM agents and tool use → Agent threats, guardrails and governance

also in 1 more
Abstract (arXiv)

Coding agents powered by large language models are becoming central modules of modern IDEs, helping users perform complex tasks by invoking tools. While powerful, tool invocation opens a substantial attack surface. Prior work has demonstrated attacks against general-purpose and domain-specific agents, but none have focused on the security risks of tool invocation in coding agents. To fill this gap, we conduct the first systematic red-teaming of six popular real-world coding agents: Cursor, Claude Code, Copilot, Windsurf, Cline, and Trae. Our red-teaming proceeds in two phases. In Phase 1, we perform prompt leakage reconnaissance to recover system prompts. We discover a general vulnerability, ToolLeak, which allows malicious prompt exfiltration through benign argument retrieval during tool invocation. In Phase 2, we hijack the agent's tool-invocation behavior using a novel two-channel prompt injection in the tool description and return values, achieving remote code execution (RCE). We adaptively construct payloads using security information leaked in Phase 1. In emulation across five backends, our method outperforms baselines on Claude-Sonnet-4, Claude-Sonnet-4.5, Grok-4, and GPT-5. On real agents, our approach succeeds on 19 of 25 agent-LLM pairs, achieving leakage on every agent using Claude and Grok backends. For tool-invocation hijacking, we obtain RCE on every tested agent-LLM pair, with our two-channel method delivering the highest success rate. We provide case studies on Cursor and Claude Code, analyze security guardrails of external and built-in tools, and conclude with practical defense recommendations.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 46 of its 91 references that are in the map. First posted on arXiv 6 Sep 2025.

Builds on

46 of 91 references.

Closest ancestors · 14

Same direction, newest first

  1. 2025MCPTox: A Benchmark for Tool Poisoning Attack on Real-World MCP Servers19 Aug
  2. 2025When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents12 Jul
  3. 2025Simple Prompt Injection Attacks Can Leak Personal Data Observed by LLM Agents During Task Execution1 Jun
  4. 2025AgentVigil: Generic Black-Box Red-teaming for Indirect Prompt Injection against LLM Agents9 May
  5. 2025Prompt Injection Attack to Tool Selection in LLM Agents28 Apr
  6. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injection19 Apr
  7. 2025StruPhantom: Evolutionary Injection Attacks on Black-Box Tabular Agents Powered by Large Language Models14 Apr
  8. 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploits2 Apr
  9. 2025Adaptive Attacks Break Defenses Against Indirect Prompt Injection Attacks on LLM Agents27 Feb
  10. 2024From Allies to Adversaries: Manipulating LLM Tool-Calling through Adversarial Injection13 Dec
  11. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agents12 Nov
  12. 2024AdvAgent: Controllable Blackbox Red-teaming on Web Agents22 Oct
2 more
  1. 2024AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents19 Jun
  2. 2024InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents5 Mar

Other references · 32

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,383
  4. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  5. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  6. 2023Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injectioncited by 1,339
  7. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 760
  8. 2023Prompt Injection attack against LLM-integrated Applicationscited by 568
  9. 2023Detecting Language Model Attacks with Perplexitycited by 411
  10. 2023Formalizing and Benchmarking Prompt Injection Attacks and Defensescited by 328
  11. 2024StruQ: Defending Against Prompt Injection with Structured Queriescited by 325
  12. 2023Tool Learning with Foundation Modelscited by 302
20 more
  1. 2024AutoCodeRover: Autonomous Program Improvementcited by 272
  2. 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
  3. 2024RepairAgent: An Autonomous, LLM-Based Agent for Program Repaircited by 225
  4. 2024Tool Learning with Large Language Models: A Surveycited by 221
  5. 2024ToolACE: Winning the Points of LLM Function Callingcited by 204
  6. 2024SecAlign: Defending Against Prompt Injection with Preference Optimizationcited by 179
  7. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  8. 2024PLeak: Prompt Leaking Attacks against Large Language Model Applicationscited by 113
  9. 2023Jina Embeddings 2: 8192-Token General-Purpose Text Embeddings for Long Documentscited by 103
  10. 2024ToolSword: Unveiling Safety Issues of Large Language Models in Tool Learning Across Three Stagescited by 77
  11. 2023Making Language Models Better Tool Learners with Execution Feedbackcited by 69
  12. 2023Confucius: Iterative Tool Learning from Introspection Feedback by Easy-to-Difficult Curriculumcited by 64
  13. 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 62
  14. 2025Enterprise-Grade Security for the Model Context Protocol (MCP): Frameworks and Mitigation Strategiescited by 42
  15. 2025MCP-Zero: Active Tool Discovery for Autonomous LLM Agentscited by 32
  16. 2024HijackRAG: Hijacking Attacks against Retrieval-Augmented Large Language Modelscited by 27
  17. 2024Prompt Leakage effect and defense strategies for multi-turn LLM interactionscited by 23
  18. 2025Real AI Agents with Fake Memories: Fatal Context Manipulation Attacks on Web3 Agentscited by 23
  19. 2024Enhancing Function-Calling Capabilities in LLMs: Strategies for Prompt Formats, Data Integration, and Multilingual Translationcited by 14
  20. 2025System Prompt Extraction Attacks and Defenses in Large Language Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025RedCodeAgent: Automatic Red-teaming Agent against Diverse Code Agents
  2. 2026Prompt Injection Attacks on Agentic Coding Assistants: A Systematic Analysis of Vulnerabilities in Skills, Tools, and Protocol Ecosystems
  3. 2026Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
  4. 2026MonitoringBench: Semi-Automated Red-Teaming for Agent Monitoring
  5. 2025Takedown: How It's Done in Modern Coding Agent Exploits
  6. 2026PI-Hunter: Automated Red-Teaming for Exposing and Localizing Prompt Injections
  7. 2026SIR: Self-improving Red-teaming for Compute Use Agents
  8. 2026Execution-Grounded Security Testing for Coding Agents in Software Engineering Pipelines