mlsanityData to 7 Oct 2026

Behind Harmful Compliance: Behavioral and Mechanistic Divergence Across LLM Jailbreaks

arXiv 2604.18510 ↗First posted 20 Apr 2026cs.CR cs.AI cs.CLcited by 0

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

Open-weight language models can be rendered unsafe through several parameter-level interventions, yet models with matched harmful compliance can exhibit fundamentally different failure modes. We compare harmful supervised fine-tuning (SFT), harmful reinforcement learning with verifiable rewards (RLVR), and refusal-feature abliteration in Qwen2.5-7B and Llama-3.1-8B using harmfulness, capability, self-audit, safety reflection, representation similarity, and refusal-direction repair. All three routes reach near-ceiling harmfulness, but SFT causes the broadest capability loss and representational drift; abliteration yields localized, family-dependent refusal-feature suppression; and RLVR largely preserves base-model capability, explicit safety judgments, and representation geometry while retargeting behavior toward compliance. RLVR models consequently remain unusually responsive to safety-reflection prompts despite complying under direct prompting. We further find that harmful RLVR induces capability-blind compliance: models claim to complete unavailable actions and fabricate information about nonexistent entities. Targeted RLVR calibration substantially reduces this false acceptance without restoring safety or degrading general capability. These results show that harmful compliance, harm recognition, and capability awareness are separable behavioral axes, and that self-audit and hallucination patterns are not necessarily robust safety signals under adaptive post-training.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 18 of its 29 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

18 of 29 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Safety Alignment as Continual Learning: Mitigating the Alignment Tax via Orthogonal Gradient Projection8 Feb
  2. 2026GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Prompt5 Feb
  3. 2025HarmRLVR: Weaponizing Verifiable Rewards for Harmful LLM Alignment17 Oct
  4. 2024No Two Devils Alike: Unveiling Distinct Mechanisms of Fine-tuning Attacks25 May
  5. 2023LoRA Fine-tuning Efficiently Undoes Safety Training in Llama 2-Chat 70B31 Oct
  6. 2023Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!5 Oct

Other references · 12

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  4. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  5. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  6. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  7. 2023Catastrophic Jailbreak of Open-source LLMs via Exploiting Generationcited by 439
  8. 2024AIR-Bench 2024: A Safety Benchmark Based on Risk Categories from Regulations and Policiescited by 69
  9. 2025Persistent Instability in LLM's Personality Measurements: Effects of Scale, Reasoning, and Conversation Historycited by 25
  10. 2024"Not Aligned" is Not "Malicious": Being Careful about Hallucinations of Large Language Models' Jailbreakcited by 23
  11. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  12. 2025Mitigating the Safety Alignment Tax with Null-Space Constrained Policy Optimizationcited by 11

Built on by

Not cited yet.