mlsanityData to 14 Oct 2026

SecDrift: Measuring Sector-Conditioned Security Drift in AI-Generated Code

arXiv 2607.25225 ↗First posted 28 Jul 2026cs.CR cs.LG cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

LLMs are increasingly used for code generation in critical infrastructure, yet the security effect of domain-specific prompting is understudied. We present SecDrift, a benchmark measuring sector-conditioned security drift: the change in static-analysis vulnerability rates when prompts are conditioned on industry contexts versus neutral baselines. We evaluate 7 LLMs (6 producing analyzable code) across 8 CISA critical infrastructure sectors and 9 CWE categories with 5 replicates (5,355 evaluations), using a 5-dimension transformation with a matched-baseline condition that holds the task fixed while substituting only domain terminology. Industry prompts naively appear more secure (14.0% vs. 11.4%, -2.7pp), but the gap is not statistically significant (Fisher's exact p = 0.24, Cohen's h = -0.08) and is a composition artifact of two CWE categories: excluding CWE-502 and CWE-22 eliminates and slightly reverses it (+0.4pp, p = 1.00). A mixed-effects logistic regression confirms sector identity is not a moderator and localizes the only detectable condition effect to those two vulnerability types. 0 of 8 sectors show drift distinguishable from baseline, corrected or uncorrected (|h| < 0.15). A placebo on two non-CISA sectors (e-commerce, online education) reproduces the CISA industry rate almost exactly (10.5% vs. 11.4%, p = 0.63): the small pooled pattern reflects generic industry-framing specificity, not critical-infrastructure identity. In contrast, model selection has a large and consistent effect: among full-output models vulnerability rates range from 11.6% to 16.1%, and these differences persist across conditions. Model choice, not prompt framing, is the more reliable security lever. We release the framework, prompts, generated code, findings, human-validation verdicts, and analysis scripts.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 7 of its 14 references that are in the map. First posted on arXiv 28 Jul 2026.

Builds on

7 of 14 references.

References · 7

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  3. 2021Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributionscited by 431
  4. 2023Large Language Models for Code: Security Hardening and Adversarial Testingcited by 155
  5. 2022Lost at C: A User Study on the Security Implications of Large Language Model Code Assistantscited by 91
  6. 2023LLMSecEval: A Dataset of Natural Language Prompts for Security Evaluationscited by 62
  7. 2025CASTLE: Benchmarking Dataset for Static Code Analyzers and LLMs towards CWE Detectioncited by 10

Built on by

Not cited yet.