# How Secure is Secure Code Generation? Adversarial Prompts Put LLM Defenses to the Test

arXiv: https://arxiv.org/abs/2601.07084 · HTML: https://mlsanity.com/paper/2601.07084/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Jailbreaking, one of 12 directions in LLM safety and security.
- Builds on 23 of its 39 references that are in the map.
- First posted on arXiv 11 Jan 2026.

## Placed in

- LLM safety and security / [Jailbreaking](https://mlsanity.com/safety/jailbreaking/)

## Builds on (most cited first, 23 of 23)

- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction](https://mlsanity.com/paper/2402.18104/) (2024)
- [Natural Attack for Pre-trained Models of Code](https://mlsanity.com/paper/2201.08698/) (2022)
- [Instruction Tuning for Secure Code Generation](https://mlsanity.com/paper/2402.09497/) (2024)
- [TrojanPuzzle: Covertly Poisoning Code-Suggestion Models](https://mlsanity.com/paper/2301.02344/) (2023)
- [CodeAttack: Code-Based Adversarial Attacks for Pre-trained Programming Language Models](https://mlsanity.com/paper/2206.00052/) (2022)
- [An LLM-Assisted Easy-to-Trigger Backdoor Attack on Code Completion Models: Injecting Disguised Vulnerabilities against Strong Detection](https://mlsanity.com/paper/2406.06822/) (2024)
- [Constrained Decoding for Secure Code Generation](https://mlsanity.com/paper/2405.00218/) (2024)
- [Vulnerabilities in AI Code Generators: Exploring Targeted Data Poisoning Attacks](https://mlsanity.com/paper/2308.04451/) (2023)
- [Toward Secure Code Generation: Bridging Correctness and Security via Task-Adaptive Vulnerability Modeling and Execution-Based Benchmarking](https://mlsanity.com/paper/2407.02395/) (2024)
- [PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)](https://mlsanity.com/paper/2409.12699/) (2024)
- [Security Attacks on LLM-based Code Completion Tools](https://mlsanity.com/paper/2408.11006/) (2024)
- [HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data](https://mlsanity.com/paper/2409.06446/) (2024)
- [On Extracting Specialized Code Abilities from Large Language Models: A Feasibility Study](https://mlsanity.com/paper/2303.03012/) (2023)
- [Gotcha! This Model Uses My Code! Evaluating Membership Leakage Risks in Code Models](https://mlsanity.com/paper/2310.01166/) (2023)
- [SecCoder: Towards Generalizable and Robust Secure Code Generation](https://mlsanity.com/paper/2410.01488/) (2024)
- [Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation](https://mlsanity.com/paper/2506.23034/) (2025)
- [How Important are Good Method Names in Neural Code Generation? A Model Robustness Perspective](https://mlsanity.com/paper/2211.15844/) (2022)
- [SABER: Model-agnostic Backdoor Attack on Chain-of-Thought in Neural Code Generation](https://mlsanity.com/paper/2412.05829/) (2024)
- [Code Difference Guided Adversarial Example Generation for Deep Code Models](https://mlsanity.com/paper/2301.02412/) (2023)

## Built on by (most cited first, 4 of 4)

- [Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs](https://mlsanity.com/paper/2605.29737/) (2026)
- [Do Not Copy/Paste: Soft Barriers for Copying in AI-Assisted Programming](https://mlsanity.com/paper/2608.22638/) (2026)
- [Evaluating Inference-Time Defenses Against Package Hallucination in LLM-Generated Code](https://mlsanity.com/paper/2608.22652/) (2026)
- [Security in LLM-Assisted Coding: A Systematic Literature Review](https://mlsanity.com/paper/2610.15482/) (2026)

## Abstract (arXiv)

Recent secure code generation methods, using vulnerability-aware fine-tuning, prefix-tuning, and prompt optimization, claim to prevent LLMs from producing insecure code. However, their robustness under adversarial conditions remains untested, and current evaluations decouple security from functionality, potentially inflating reported gains. We present the first systematic adversarial audit of state-of-the-art secure code generation methods (SVEN, SafeCoder, PromSec). We subject them to realistic prompt perturbations such as paraphrasing, cue inversion, and context manipulation that developers might inadvertently introduce or adversaries deliberately exploit. To enable fair comparison, we evaluate all methods under consistent conditions, jointly assessing security and functionality using multiple analyzers and executable tests. Our findings reveal critical robustness gaps: static analyzers overestimate security by 7 to 21 times, with 37 to 60% of ``secure'' outputs being non-functional. Under adversarial conditions, true secure-and-functional rates collapse to 3 to 17%. Based on these findings, we propose best practices for building and evaluating robust secure code generation methods. Our code is available.