mlsanityData to 14 Oct 2026

Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

arXiv 2607.23088 ↗First posted 25 Jul 2026cs.CR cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are widely used for code generation, yet their security behavior in realistic development workflows remains underexplored. Existing benchmarks often rely on explicitly specified security requirements, failing to capture real-world scenarios where prompts are frequently ambiguous or incomplete. In this paper, we adopt a developer-centric perspective and identify three representative risk scenarios that commonly lead to security vulnerabilities in LLM-generated code: Ambiguous Requirements, Under-Specified Operational Context, and Security--Functionality Conflict. Based on these scenarios, we construct a large-scale benchmark comprising 2,700 test cases, enabling fine-grained evaluation of LLM security under realistic conditions. Extensive evaluation of eight state-of-the-art LLMs reveals that all models exhibit average vulnerability rates exceeding 56% across risk scenarios. We further demonstrate that security-aware prompting can substantially mitigate these risks, achieving up to 45% improvement.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 8 references that are in the map. First posted on arXiv 25 Jul 2026.

Builds on

4 of 8 references.

References · 4

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  4. 2022Do Users Write More Insecure Code with AI Assistants?cited by 192

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
  2. 2024From Vulnerabilities to Remediation: A Systematic Literature Review of LLMs in Code Security
  3. 2025Explicit Vulnerability Generation with LLMs: An Investigation Beyond Adversarial Attacks
  4. 2026Enhancing Reliability in LLM-Based Secure Code Generation
  5. 2025Rethinking the Evaluation of Secure Code Generation
  6. 2025When Code Crosses Borders: A Security-Centric Study of LLM-based Code Translation ↗
  7. 2025Smoke and Mirrors: Jailbreaking LLM-based Code Generation via Implicit Malicious Prompts
  8. 2026Security-by-Design for LLM-Based Code Generation: Leveraging Internal Representations for Concept-Driven Steering Mechanisms