mlsanityData to 14 Oct 2026

Enhancing Reliability in LLM-Based Secure Code Generation

arXiv 2605.24300 ↗First posted 22 May 2026cs.CR cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) are widely used for code generation, but their security reliability remains inconsistent across languages and prompting strategies. Existing prompt engineering improves functional correctness but rarely ensures consistent security outcomes. We introduce the textitMitigation-Aware Chain-of-Thought (MA-CoT) framework, which embeds task-specific CWE mitigation guidance and language-aware safeguards to reduce recurring vulnerabilities in generated code. We evaluate MA-CoT across three LLMs (gpt-5, claude-4.5, gemini-2.5), three programming languages (C, Java, Python), and four prompting strategies (Vanilla, Zero-shot, CoT, MA-CoT) on a 200-task primary dataset, with external validation on LLMSecEval. Using static analysis with expert validation, MA-CoT reduces total security findings from 92 to 39 (57.6%) on the primary dataset and from 73 to 4 (94.5%) on LLMSecEval. High-severity findings (Blocker + Critical) drop from 90 to 39 (56.7%) and from 45 to 2 (95.6%), respectively. Across both datasets, MA-CoT is the only strategy that consistently improves security reliability; Zero-shot and CoT are less reliable and may increase vulnerability, especially in C. We further introduce a strict layered attribution of vulnerability drivers (language-core vs. stack layers) and show that residual risk concentrates in hardening-oriented patterns (e.g., OS- and toolchain-dependent), motivating secure-by-construction primitives alongside prompting.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 35 of its 49 references that are in the map. First posted on arXiv 22 May 2026.

Builds on

35 of 49 references.

References · 35

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  4. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  5. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 431
  6. 2023Language Models can Solve Computer Taskscited by 372
  7. 2023Self-planning Code Generation with Large Language Modelscited by 124
  8. 2023LLMs Cannot Reliably Identify and Reason About Security Vulnerabilities (Yet?): A Comprehensive Evaluation, Framework, and Benchmarkscited by 97
  9. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 91
  10. 2023Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPTcited by 71
  11. 2022Is GitHub's Copilot as Bad as Humans at Introducing Vulnerabilities in Code?cited by 67
  12. 2024Why Does the Effective Context Length of LLMs Fall Short?cited by 63
23 more
  1. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 62
  2. 2025CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generationcited by 59
  3. 2025BaxBench: Can LLMs Generate Correct and Secure Backends?cited by 41
  4. 2023SALLM: Security Assessment of Generated Codecited by 39
  5. 2023Security Weaknesses of Copilot-Generated Code in GitHub Projects: An Empirical Studycited by 38
  6. 2024Prompting Techniques for Secure Code Generation: A Systematic Investigationcited by 34
  7. 2024Chain-of-Thought Prompting of Large Language Models for Discovering and Fixing Software Vulnerabilitiescited by 31
  8. 2024Using AI Assistants in Software Development: A Qualitative Study on Security Practices and Concernscited by 28
  9. 2025Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applicationscited by 26
  10. 2024How secure is AI-generated Code: A Large-Scale Comparison of Large Language Modelscited by 24
  11. 2024PromSec: Prompt Optimization for Secure Generation of Functional Source Code with Large Language Models (LLMs)cited by 19
  12. 2025Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysiscited by 19
  13. 2025Give LLMs a Security Course: Securing Retrieval-Augmented Code Generation via Knowledge Injectioncited by 16
  14. 2023A User-centered Security Evaluation of Copilotcited by 15
  15. 2025Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Modelscited by 12
  16. 2025Rethinking the Evaluation of Secure Code Generationcited by 10
  17. 2024DeVAIC: A Tool for Security Assessment of AI-generated Codecited by 8
  18. 2025RESCUE: Retrieval Augmented Secure Code Generationcited by 7
  19. 2024"You still have to study" -- On the Security of LLM generated codecited by 3
  20. 2025Secure-Instruct: An Automated Pipeline for Synthesizing Instruction-Tuning Datasets Using LLMs for Secure Code Generationcited by 2
  21. 2019Statically Detecting Vulnerabilities by Processing Programming Languages as Natural Languagescited by 1
  22. 2025Secure Code Generation at Scale with Reflexioncited by 1
  23. 2025LLM-CSEC: Empirical Evaluation of Security in C/C++ Code Generated by Large Language Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Securing AI-Generated Code: A Just-in-Time Vulnerability Detection and Remediation Pipelinecited by 0
  2. 2026Learning from Failures: A Failure-Driven Prompt Refinement for LLM-Based Vulnerability Analysiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
  2. 2026Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios
  3. 2025A Mixture of Linear Corrections Generates Secure Code
  4. 2025Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks
  5. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security
  6. 2026Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms
  7. 2025Smoke and Mirrors: Jailbreaking LLM-based Code Generation via Implicit Malicious Prompts
  8. 2026Inference-Time Safety For Code LLMs Via Retrieval-Augmented Revision