mlsanityData to 14 Oct 2026

Minimal Prompt Perturbations Lead to Code Vulnerabilities: Prompt Fragility and Hidden-State Signals in Coding LLMs

arXiv 2605.29737 ↗First posted 28 May 2026cs.CR cs.CL cs.SEcited by 2

Not on the map. Why?

Abstract (arXiv)

LLM-based coding assistants are seeing rapid adoption, offering substantial gains in developer productivity. As organizations increasingly ship code these agents produce, the security of that code becomes critical. Prior work has shown that minor prompt perturbations degrade the functional correctness of LLM-generated code, but whether they also compromise code security has remained unstudied. We apply token-level mutations to prompts across three models and five programming languages, and show that mutations as small as a single-character change can flip generated code from secure to vulnerable. Probing the models' hidden states reveals that this fragility is partially encoded in prompt representations, but unevenly so. Input-handling vulnerabilities, where the model omits validation or sanitization, are more predictable (mean AUC 0.753) than secure-defaults vulnerabilities, where insecure code stems from one local choice such as a weak algorithm or unsafe parameter (mean AUC 0.674). These results show that the threat model for LLM-assisted coding extends beyond prompt injection to ordinary prompt variation, and indicate that input-handling flaws can be caught before generation while secure-defaults flaws require intervention during decoding.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 13 of its 17 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

13 of 17 references.

References · 13

  1. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  2. 2024A Survey on Large Language Models for Code Generationcited by 658
  3. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  4. 2020Information-Theoretic Probing for Linguistic Structurecited by 207
  5. 2022ReCode: Robustness Evaluation of Code Generation Modelscited by 103
  6. 2025CWEval: Outcome-driven Evaluation on Functionality and Security of LLM Code Generationcited by 59
  7. 2024NLPerturbator: Studying the Robustness of Code LLMs to Natural Language Variationscited by 20
  8. 2025Who is using AI to code? Global diffusion and impact of generative AIcited by 18
  9. 2025Correctness Assessment of Code Generated by Large Language Models Using Internal Representationscited by 17
  10. 2023Exploring the Robustness of Large Language Models for Solving Programming Problemscited by 16
  11. 2025A Multi-Language Perspective on the Robustness of LLM Code Generationcited by 6
  12. 2025CREME: Robustness Enhancement of Code LLMs via Layer-Aware Model Editingcited by 5
1 more
  1. 2026How Secure is Secure Code Generation? Adversarial Prompts Put LLM Defenses to the Testcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025A Mixture of Linear Corrections Generates Secure Code
  2. 2026An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
  3. 2025Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks
  4. 2026SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
  5. 2026Insecure Coding Preferences in Long-Term Memory: Security Risks for LLM-based Code Generation ↗
  6. 2026Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios
  7. 2025Benchmarking Prompt Engineering Techniques for Secure Code Generation with GPT Models
  8. 2024MaPPing Your Model: Assessing the Impact of Adversarial Attacks on LLM-based Programming Assistants