Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders
On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks
also in 1 more
Abstract (arXiv)
Safety alignment in large language models (LLMs), particularly for cybersecurity tasks, primarily focuses on preventing misuse. While this approach reduces direct harm, it obscures a complementary failure mode: denial of assistance to legitimate defenders. We study Defensive Refusal Bias -- the tendency of safety-tuned frontier LLMs to refuse assistance for authorized defensive cybersecurity tasks when those tasks include similar language to an offensive cyber task. Based on 2,390 real-world examples from the National Collegiate Cyber Defense Competition (NCCDC), we find that LLMs refuse defensive requests containing security-sensitive keywords at 2.72× the rate of semantically equivalent neutral requests (p < 0.001). The highest refusal rates occur in the most operationally critical tasks: system hardening (43.8%) and malware analysis (34.3%). Interestingly, explicit authorization, where the user directly instructs the model that they have authority to complete the target task, increases refusal rates, suggesting models interpret justifications as adversarial rather than exculpatory. These findings are urgent for interactive use and critical for autonomous defensive agents, which cannot rephrase refused queries or retry. Our findings suggest that current LLM cybersecurity alignment relies on semantic similarity to harmful content rather than reasoning about intent or authorization. We call for mitigations that analyze intent to maximize defensive capabilities while still preventing harmful compliance.
Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 16 of its 20 references that are in the map. First posted on arXiv 1 Mar 2026.
Builds on
16 of 20 references.
Closest ancestors · 2
Same direction, newest first
- 2024OR-Bench: An Over-Refusal Benchmark for Large Language Models31 May
- 2023Universal and Transferable Adversarial Attacks on Aligned Language Models27 Jul
Other references · 14
- 2022Training language models to follow instructions with human feedbackcited by 15,971
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,651
- 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
- 2023Identifying the Risks of LM Agents with an LM-Emulated Sandboxcited by 548
- 2025Constitutional Classifiers: Defending against Universal Jailbreaks across Thousands of Hours of Red Teamingcited by 181
- 2023DiverseVul: A New Vulnerable Source Code Dataset for Deep Learning Based Vulnerability Detectioncited by 102
- 2024When LLMs Meet Cybersecurity: A Systematic Literature Reviewcited by 71
- 2024CTIBench: A Benchmark for Evaluating LLMs in Cyber Threat Intelligencecited by 62
Built on by
Citing papers per year: 2026: 10.
10 papers.
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?cited by 17
- 2026Agent Memory Is a Surface for Endogenous Authorization Launderingcited by 5
- 2026Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysiscited by 1
- 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Responsecited by 1
- 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustnesscited by 1
- 2026Poster: ClawdGo: Endogenous Security Awareness Training for Autonomous AI Agentscited by 0
- 2026Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackerscited by 0
- 2026LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Contextcited by 0
- 2026Same Request, Different Boundary: Evaluating Cybersecurity Assistance across Conversational Contextscited by 0
- 2026Pretext: Defeating Malicious Skill Detection Frameworks for AI Agentscited by 0
- 2026Pretext: Defeating Malicious Skill Detection Frameworks for AI Agents30 Sep 2026
- 2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustness3 Sep 2026
- 2026Agent Memory Is a Surface for Endogenous Authorization Laundering1 Sep 2026
- 2026Same Request, Different Boundary: Evaluating Cybersecurity Assistance across Conversational Contexts1 Sep 2026
- 2026Cyber-Capable AI Agents: Vulnerabilities, Evaluation Containment, and Defensive Response28 Jul 2026
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?11 Jul 2026
- 2026Beyond Refusal: A Same-Lineage Study of Aligned and Abliterated LLMs for Vulnerability Analysis7 Jul 2026
- 2026LLMs Prompted for Legal Context Object More: Overrefusal from Small On-Premises LLMs in Criminal Legal Context23 Jun 2026
- 2026Honeyquest for LLMs: Rethinking Cyber Deception for AI Attackers19 Jun 2026
- 2026Poster: ClawdGo: Endogenous Security Awareness Training for Autonomous AI Agents27 Apr 2026
Similar, not cited
Close in wording, no citation either way
- 2026Not All Refusals Are Equal: How Safety Alignment Fails Cybersecurity at Scale
- 2026A Content-Based Framework for Cybersecurity Refusal Decisions in Large Language Models
- 2026Ablating Safety: Mechanisms for Removing Alignment in Language Models for Security Applications
- 2026A New Framework for Cybersecurity Refusals in AI Agents
- 2026Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment
- 2026From Refusal Geometry to Safety Geometry: Harmfulness--Refusal Coupling under Dynamic Adversarial Fine-Tuning
- 2026Understanding the Effects of Safety Unalignment on Large Language Models
- 2026The Autonomy Tax: Defense Training Breaks LLM Agents