Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis
Not on the map. Why?
Abstract (arXiv)
Large language model (LLM)-assisted software security operates at a difficult boundary: the vulnerability-analysis terminology needed for legitimate code review, triage, and repair can closely resemble terminology associated with misuse. Existing safety and cybersecurity evaluations are difficult to interpret in this setting because they often compare unrelated model families, thereby conflating safety behavior with differences in architecture, scale, training data, and deployment. To isolate this factor, we study safety state: whether refusal behavior remains intact (Aligned) or has been refusal-ablated (Abliterated) within same-lineage models. We ask how this safety state affects defensive utility across software-security workflows. We compare aligned instruction-tuned models with publicly released refusal-ablated descendants from two model families, Gemma and Qwen. We evaluate Aligned and Abliterated states on vulnerability detection, CWE attribution, vulnerable-line localization, root-cause localization, and executable patch validation. We further treat prompt wording as a controlled framing dimension: prompts begin with neutral code-review language, add authorization context, and vary the density of cybersecurity terminology. In a Gemma-based Java/Vul4J repair-validation study, Abliterated achieves higher early-stage validation rates, with 67.8%, 65.0%, and 32.8% of patches judged usable, successfully applied, and successfully compiled, respectively, compared with 29.9%, 24.9%, and 9.0% for Aligned. In the Qwen pair, Abliterated improves localization performance, increasing line-level F1 from 2.08% to 3.91% and Top-1 accuracy from 4.10% to 6.95%. These findings suggest that evaluations of LLM-based security assistants should jointly measure whether models respond, whether their usable responses are correct, and whether their outputs remain actionable across the engineering workflow.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 41 of its 54 references that are in the map. First posted on arXiv 7 Jul 2026.
Builds on
41 of 54 references.
References · 41
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
- 2023Code Llama: Open Foundation Models for Codecited by 2,105
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
- 2022Competition-Level Code Generation with AlphaCodecited by 1,427
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
- 2020CodeBERT: A Pre-Trained Model for Programming and Natural Languagescited by 1,286
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
29 more
- 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,154
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
- 2021CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generationcited by 890
- 2024Gemma: Open Models Based on Gemini Research and Technologycited by 889
- 2023StarCoder: may the source be with you!cited by 790
- 2021CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generationcited by 732
- 2023WizardCoder: Empowering Code Large Language Models with Evol-Instructcited by 728
- 2020GraphCodeBERT: Pre-training Code Representations with Data Flowcited by 600
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 514
- 2024Agentless: Demystifying LLM-based Software Engineering Agentscited by 400
- 2021Unified Pre-training for Program Understanding and Generationcited by 361
- 2022UniXcoder: Unified Cross-Modal Pre-training for Code Representationcited by 355
- 2024AutoCodeRover: Autonomous Program Improvementcited by 272
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Modelscited by 222
- 2024Vulnerability Detection with Code Language Models: How Far Are We?cited by 133
- 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 98
- 2021CVEfixes: Automated Collection of Vulnerabilities and Their Fixes from Open-Source Softwarecited by 89
- 2024CyberSecEval 2: A Wide-Ranging Cybersecurity Evaluation Suite for Large Language Modelscited by 84
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 70
- 2024CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligencecited by 61
- 2024CYBERSECEVAL 3: Advancing the Evaluation of Cybersecurity Risks and Capabilities in Large Language Modelscited by 56
- 2024Surgical, Cheap, and Flexible: Mitigating False Refusal in Language Models via Single Vector Ablationcited by 48
- 2024MegaVul: A C/C++ Vulnerability Dataset with Comprehensive Code Representationcited by 27
- 2024A Case Study of LLM for Automated Vulnerability Repair: Assessing Impact of Reasoning and Patch Validation Feedbackcited by 22
- 2025FORTRESS: Frontier Risk Evaluation for National Security and Public Safetycited by 20
- 2025PATCHEVAL: A New Benchmark for Evaluating LLMs on Patching Real-World Vulnerabilitiescited by 19
- 2025SoK: Automated Vulnerability Repair: Methods, Tools, and Assessmentscited by 11
- 2026Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenderscited by 10
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications
- 2026A Validated Prompt Bank for Malicious Code Generation: Separating Executable Weapons from Security Knowledge in 1,554 Consensus-Labeled Prompts
- 2026An Empirical Evaluation of LLM-Generated Code Security Across Prompting Methods
- 2026Measuring Safety Alignment Effects in Autonomous Security Agents
- 2026Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
- 2026Mind the GAP: Text Safety Does Not Transfer to Tool-Call Safety in LLM Agents
- 2025The Hidden Dimensions of LLM Alignment: A Multi-Dimensional Analysis of Orthogonal Safety Directions
- 2026Routing Ceilings Are Domain-Independent: Structural Prior Injection in Code Security Vulnerability Detection