# LLMs + Security = Trouble

arXiv: https://arxiv.org/abs/2602.08422 · HTML: https://mlsanity.com/paper/2602.08422/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 41 of its 50 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 30 of 41)

- [Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions](https://mlsanity.com/paper/2108.09293/) (2021)
- [DiffuCoder: Understanding and Improving Masked Diffusion Models for Code Generation](https://mlsanity.com/paper/2506.20639/) (2025)
- [Large Language Models for Code: Security Hardening and Adversarial Testing](https://mlsanity.com/paper/2302.05319/) (2023)
- [Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation](https://mlsanity.com/paper/2403.06988/) (2024)
- [Gradient-Based Constrained Sampling from Language Models](https://mlsanity.com/paper/2205.12558/) (2022)
- [VeriGuard: Enhancing LLM Agent Safety via Verified Code Generation](https://mlsanity.com/paper/2510.05156/) (2025)
- [Flexible and Efficient Grammar-Constrained Decoding](https://mlsanity.com/paper/2502.05111/) (2025)
- [A New Era in Software Security: Towards Self-Healing Software via Large Language Models and Formal Verification](https://mlsanity.com/paper/2305.14752/) (2023)
- [Constrained Decoding for Secure Code Generation](https://mlsanity.com/paper/2405.00218/) (2024)
- [Agentic AI Software Engineers: Programming with Trust](https://mlsanity.com/paper/2502.13767/) (2025)
- [APPATCH: Automated Adaptive Prompting Large Language Models for Real-World Software Vulnerability Patching](https://mlsanity.com/paper/2408.13597/) (2024)
- [Security and Quality in LLM-Generated Code: A Multi-Language, Multi-Model Analysis](https://mlsanity.com/paper/2502.01853/) (2025)
- [HexaCoder: Secure Code Generation via Oracle-Guided Synthetic Training Data](https://mlsanity.com/paper/2409.06446/) (2024)
- [Exploring the Potential of Diffusion Large Language Models in Code Generation](https://mlsanity.com/paper/2509.11252/) (2025)
- [ChopChop: a Programmable Framework for Semantically Constraining the Output of Language Models](https://mlsanity.com/paper/2509.00360/) (2025)
- [Efficient Avoidance of Vulnerabilities in Auto-completed Smart Contract Code Using Vulnerability-constrained Decoding](https://mlsanity.com/paper/2309.09826/) (2023)
- [Assessing Correctness in LLM-Based Code Generation via Uncertainty Estimation](https://mlsanity.com/paper/2502.11620/) (2025)
- [Guiding AI to Fix Its Own Flaws: An Empirical Study on LLM-Driven Secure Code Generation](https://mlsanity.com/paper/2506.23034/) (2025)
- [Systems Security Foundations for Agentic Computing](https://mlsanity.com/paper/2512.01295/) (2025)
- [Artificial-Intelligence Generated Code Considered Harmful: A Road Map for Secure and High-Quality Code Generation](https://mlsanity.com/paper/2409.19182/) (2024)
- [DiffuGuard: How Intrinsic Safety is Lost and Found in Diffusion Large Language Models](https://mlsanity.com/paper/2509.24296/) (2025)
- [TreeDiff: AST-Guided Code Generation with Diffusion LLMs](https://mlsanity.com/paper/2508.01473/) (2025)
- [Diffusion On Syntax Trees For Program Synthesis](https://mlsanity.com/paper/2405.20519/) (2024)
- [Low-Cost and Comprehensive Non-textual Input Fuzzing with LLM-Synthesized Input Generators](https://mlsanity.com/paper/2501.19282/) (2025)
- [Rethinking the Evaluation of Secure Code Generation](https://mlsanity.com/paper/2503.15554/) (2025)
- [A Neurosymbolic Approach to Natural Language Formalization and Verification](https://mlsanity.com/paper/2511.09008/) (2025)
- [Towards Formal Verification of LLM-Generated Code from Natural Language Prompts](https://mlsanity.com/paper/2507.13290/) (2025)
- [A Systematic Literature Review on Detecting Software Vulnerabilities with Large Language Models](https://mlsanity.com/paper/2507.22659/) (2025)
- [PoCo: Agentic Proof-of-Concept Exploit Generation for Smart Contracts](https://mlsanity.com/paper/2511.02780/) (2025)
- [RESCUE: Retrieval Augmented Secure Code Generation](https://mlsanity.com/paper/2510.18204/) (2025)

## Abstract (arXiv)

We argue that when it comes to producing secure code with AI, the prevailing "fighting fire with fire" approach -- using probabilistic AI-based checkers or attackers to secure probabilistically generated code -- fails to address the long tail of security bugs. As a result, systems may remain exposed to zero-day vulnerabilities that can be discovered by better-resourced or more persistent adversaries. While neurosymbolic approaches that combine LLMs with formal methods are attractive in principle, we argue that they are difficult to reconcile with the "vibe coding" workflow common in LLM-assisted development: unless the end-to-end verification pipeline is fully automated, developers are repeatedly asked to validate specifications, resolve ambiguities, and adjudicate failures, making the human-in-the-loop a likely point of weakness, compromising secure-by-construction guarantees. In this paper we argue that stronger security guarantees can be obtained by enforcing security constraints during code generation (e.g., via constrained decoding), rather than relying solely on post-hoc detection and repair. This direction is particularly promising for diffusion-style code models, whose approach provides a natural elegant opportunity for modular, hierarchical security enforcement, allowing us to combine lower-latency generation techniques with generating secure-by-construction code.