# Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation

arXiv: https://arxiv.org/abs/2506.23034 · HTML: https://mlsanity.com/paper/2506.23034/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2025.

## Built on by (most cited first, 13 of 13)

- [When Developer Aid Becomes Security Debt: A Systematic Analysis of Insecure Behaviors in LLM Coding Agents](https://mlsanity.com/paper/2507.09329/) (2025)
- [How Secure is Secure Code Generation? Adversarial Prompts Put LLM Defenses to the Test](https://mlsanity.com/paper/2601.07084/) (2026)
- [DeepGuard: Secure Code Generation via Multi-Layer Semantic Aggregation](https://mlsanity.com/paper/2604.09089/) (2026)
- [From Prompt to Product: A Human-Centered Benchmark of Agentic App Generation Systems](https://mlsanity.com/paper/2512.18080/) (2025)
- [Adaptive Hierarchical Evaluation of LLMs and SAST tools for CWE Prediction in Python](https://mlsanity.com/paper/2601.01320/) (2026)
- [Surgical Repair of Insecure Code Generation in LLMs](https://mlsanity.com/paper/2604.16697/) (2026)
- [LLMs + Security = Trouble](https://mlsanity.com/paper/2602.08422/) (2026)
- [Why Do Large Language Models Generate Harmful Content?](https://mlsanity.com/paper/2604.11663/) (2026)
- [Quality and Security Signals in AI-Generated Python Refactoring Pull Requests](https://mlsanity.com/paper/2605.21453/) (2026)
- [Security of LLM-generated Code: A Comparative Analysis](https://mlsanity.com/paper/2605.23091/) (2026)
- [From Evaluation to Optimisation: Hierarchy-Aware Training Signals for CWE Prediction in Python](https://mlsanity.com/paper/2607.21069/) (2026)
- [Tool-Guided Retrieval-Augmented Repair for Securing LLM-Generated C Code](https://mlsanity.com/paper/2607.21641/) (2026)
- [Interpreting and Steering for Safe and Correct Code Generation](https://mlsanity.com/paper/2608.30025/) (2026)

## Abstract (arXiv)

Large Language Models have become powerful tools for programming. However, they often overlook essential security practices, producing insecure code with vulnerabilities. Despite this risk, existing work offers limited guidance on steering LLMs toward secure code generation and lacks systematic analysis of how effectively LLMs repair vulnerable code. In this work, we investigate how LLMs can be guided to prevent and repair security vulnerabilities during code generation. Specifically, we examine whether self-generated vulnerability hints help models avoid insecure code, and evaluate how different feedback levels influence post-hoc vulnerability repair. Our study considers proprietary and open-weight models across multiple scales and uses established benchmarks covering diverse vulnerability types. Our results show that self-generated vulnerability hints meaningfully reduce insecure code, with effectiveness depending strongly on relevance and preciseness. We further find that more directive hints, which name the target weakness, explain how it could arise in the task, and specify how to avoid it, more effectively prevent vulnerable code. For post-hoc vulnerability repair, raw detection-tool feedback improves security across all models, while detailed, actionable explanations provide further gains on two of the three benchmarks, especially for models with stronger instruction-following capabilities. Yet, this explained feedback does not consistently outperform the raw feedback for the benchmark containing real-world tasks triggering multiple weaknesses.