mlsanityData to 14 Oct 2026

When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents

arXiv 2507.09329 ↗First posted 12 Jul 2025cs.AI cs.CRcited by 8

On the map: LLM agents and tool use → Agent threats, guardrails and governance

Abstract (arXiv)

LLM-based coding agents are rapidly being deployed in software development, yet their safety implications remain poorly understood. These agents, while capable of accelerating software development, may exhibit unsafe behaviors during normal operation that manifest as cybersecurity vulnerabilities. We conducted the first systematic safety evaluation of autonomous coding agents, analyzing over 12,000 actions across five state-of-the-art models (GPT-4o, GPT-4.1, Claude variants) on 93 real-world software setup tasks. Our findings reveal significant security concerns: 21% of agent trajectories contained insecure actions, with models showing substantial variation in unsafe behavior. We developed a high-precision detection system that identified four major vulnerability categories, with information exposure (CWE-200) being the most prevalent one. We also evaluated mitigation strategies including feedback mechanisms and security reminders with various effectiveness between models. GPT-4.1 demonstrated exceptional security awareness with 96.8% mitigation success.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Agent threats, guardrails and governance, one of 16 directions in LLM agents and tool use. Builds on 17 of its 33 references that are in the map. First posted on arXiv 12 Jul 2025.

Builds on

17 of 33 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025LlamaFirewall: An open source guardrail system for building secure AI agents6 May
  2. 2025AgentSpec: Customizable Runtime Enforcement for Safe and Reliable LLM Agents24 Mar
  3. 2024Agent-SafetyBench: Evaluating the Safety of LLM Agents19 Dec
  4. 2024RedCode: Risky Code Execution and Generation Benchmark for Code Agents12 Nov

Other references · 13

  1. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  2. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  3. 2022Ignore Previous Prompt: Attack Techniques For Language Modelscited by 832
  4. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  5. 2024Long-context LLMs Struggle with Long In-context Learningcited by 286
  6. 2025Securing AI Agents with Information-Flow Controlcited by 121
  7. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 97
  8. 2025Lessons from Defending Gemini Against Indirect Prompt Injectionscited by 43
  9. 2024WildFeedback: Aligning LLMs With In-situ User Interactions And Feedbackcited by 31
  10. 2024Easy Problems That LLMs Get Wrongcited by 30
  11. 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environmentscited by 18
  12. 2025Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generationcited by 13
1 more
  1. 2025Revisiting Backdoor Attacks on LLMs: A Stealthy and Practical Poisoning Framework via Harmless Inputscited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
  2. 2025Takedown: How It's Done in Modern Coding Agent Exploits
  3. 2024AutoSafeCoder: A Multi-Agent Framework for Securing LLM Code Generation through Static Analysis and Fuzz Testing
  4. 2025Shadows in the Code: Exploring the Risks and Defenses of LLM-based Multi-Agent Software Development Systems
  5. 2026MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents
  6. 2026When Passing Tests Hides Vulnerabilities: An Empirical Study of Silent Failures in Agentic Systems
  7. 2026Trust but Verify? Uncovering the Security Debt of Autonomous Coding Agents
  8. 2026Local LLM Agents as Vulnerable Runtimes:A Source-Code Audit of the Agent Runtime Layer