Data to 14 Oct 2026

LLMs Encode Harmfulness and Refusal Separately

arXiv 2507.11878 ↗First posted 16 Jul 2025cs.CLcited by 88

On the map: LLM safety and security → Safety alignment, refusal and guardrails

Abstract (arXiv)

LLMs are trained to refuse harmful instructions, but do they truly understand harmfulness beyond just refusing? Prior work has shown that LLMs' refusal behaviors can be mediated by a one-dimensional subspace, i.e., a refusal direction. In this work, we identify a new dimension to analyze safety mechanisms in LLMs, i.e., harmfulness, which is encoded internally as a separate concept from refusal. There exists a harmfulness direction that is distinct from the refusal direction. As causal evidence, steering along the harmfulness direction can lead LLMs to interpret harmless instructions as harmful, but steering along the refusal direction tends to elicit refusal responses directly without reversing the model's judgment on harmfulness. Furthermore, using our identified harmfulness concept, we find that certain jailbreak methods work by reducing the refusal signals without reversing the model's internal belief of harmfulness. We also find that adversarially finetuning models to accept harmful instructions has minimal impact on the model's internal belief of harmfulness. These insights lead to a practical safety application: The model's latent harmfulness representation can serve as an intrinsic safeguard (Latent Guard) for detecting unsafe inputs and reducing over-refusals that is robust to finetuning attacks. For instance, our Latent Guard achieves performance comparable to or better than Llama Guard 3 8B, a dedicated finetuned safeguard model, across different jailbreak methods. Our findings suggest that LLMs' internal understanding of harmfulness is more robust than their refusal decision to diverse input instructions, offering a new perspective to study AI safety.

Key facts. Cited by 88 papers in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 48 of its 55 references that are in the map. First posted on arXiv 16 Jul 2025.

Builds on

48 of 55 references.

Closest ancestors · 12

Same direction, newest first

Other references · 36

2017Attention Is All You Needcited by 60,361
2022Training language models to follow instructions with human feedbackcited by 15,971
2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
2024Qwen2 Technical Reportcited by 2,120
2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085

24 more

2023Steering Language Models With Activation Engineeringcited by 922
2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
2023The Internal State of an LLM Knows When It's Lyingcited by 659
2024How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMscited by 573
2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 560
2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
2023Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructionscited by 387
2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 231
2023Linear Representations of Sentiment in Large Language Modelscited by 163
2024Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Trackingcited by 149
2023Future Lens: Anticipating Subsequent Tokens from a Single Hidden Statecited by 121
2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
2024Robust LLM safeguarding via refusal feature adversarial trainingcited by 64
2024On Evaluating the Durability of Safeguards for Open-Weight LLMscited by 52
2024Navigating the OverKill in Large Language Modelscited by 44
2024A Language Model's Guide Through Latent Spacecited by 43
2025AdaSteer: Your Aligned LLM is Inherently an Adaptive Jailbreak Defendercited by 38
2024Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Modelscited by 37
2024ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templatescited by 30
2025Understanding Refusal in Language Models with Sparse Autoencoderscited by 29
2025COSMIC: Generalized Refusal Direction Identification in LLM Activationscited by 19

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 11; 2026: 77.

88 papers.

Sort:

By area:

2025Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMscited by 37
2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
2026There Is More to Refusal in Large Language Models than a Single Directioncited by 26
2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusalcited by 19
2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtcited by 15
2025Self-Jailbreaking: Language Models Can Reason Themselves Out of Safety Alignment After Benign Reasoning Trainingcited by 12
2025Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluationcited by 9
2026Efficient and accurate steering of Large Language Models through attention-guided feature learningcited by 8
2025Active Attacks: Red-teaming LLMs via Adaptive Environmentscited by 7
2025Unified Defense for Large Language Models against Jailbreak and Fine-Tuning Attacks in Educationcited by 5
2026Harmful Intent as a Geometrically Recoverable Feature of LLM Residual Streamscited by 4
2026SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignmentcited by 4

38 more

2025How Language Models Conflate Logical Validity with Plausibility: A Representational Analysis of Content Effectscited by 3
2025Any-Depth Alignment: Unlocking Innate Safety Alignment of LLMs to Any-Depthcited by 3
2026How Alignment Routes: Localizing, Scaling, and Controlling Policy Circuits in Language Modelscited by 3
2026Large Language Models Generate Harmful Responses Using a Distinct Mechanism, Shared Across Harm Typescited by 3
2026Probing Persona-Dependent Preferences in Language Modelscited by 3
2026A Low-Rank Subspace Analysis of LLM Interventionscited by 3
2026Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMscited by 2
2026Blind Refusal: Language Models Refuse to Help Users Evade Unjust, Absurd, and Illegitimate Rulescited by 2
2026Dynamic Adversarial Fine-Tuning Reorganizes Refusal Geometrycited by 2
2026Reflections and New Directions for Human-Centered Large Language Modelscited by 2
2026Low-Resource Safety Failures Are Action Failures, Not Representation Failurescited by 2
2026The Piggyback Hypothesis of Generalization: Explaining and Mitigating Emergent Misalignmentcited by 2
2026When Behavioral Safety Evaluation Fails: A Representation-Level Perspectivecited by 2
2026OpenSafeIntent: Evaluating Intent-Calibrated Safe Completion Across Dual-Use Prompt Setscited by 2
2026Silent Alarm: A J-Space Protocol for Comparing Danger Recognition Across Models and Quantization Levelscited by 2
2026The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicatorscited by 2
2026Do All LLMs Know When They're Being Harmful? A Reproducibility Study of Latent-Space Safety Probes Across Model Familiescited by 2
2025ASGuard: Activation-Scaling Guard to Mitigate Targeted Jailbreaking Attackcited by 1
2025Why Does Weak-OOD Help? A Further Step Towards Understanding Jailbreaking VLMscited by 1
2026The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMscited by 1
2026Persona Non Grata: Single-Method Safety Evaluation Is Incomplete for Persona-Imbued LLMscited by 1
2026Open-Weight LLM Fine-Tuning Defenses are Susceptible to Simple Attackscited by 1
2026DataShield: Safety-degrading Data Filtering for LLM Benign Instruction Fine-Tuningcited by 1
2026When Refusal Looks Safe: The Refusal-Cue Shortcut in Safety Guard Modelscited by 1
2026Online Conformal Prediction Beyond Feedbackcited by 1
2026What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Modelscited by 1
2026Beyond Shallow Alignment: How Post-Training Methods Determine Refusal Circuits And Steering Robustnesscited by 1
2026Refusal Reads Only a Slice of What the Model Knows: Harm-Keyed Routing and Its Exceptions Across Model Familiescited by 1
2026Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliterationcited by 1
2026AEGIS: Audio Endogenous Guarding via Internal Signals Against Large Audio-Language Model Jailbreakscited by 1
2026Readable Before Actionable: Causal Tracing of Indirect Prompt Injectioncited by 1
2026CoT is Not the Chain of Truth: An Empirical Internal Analysis of Reasoning LLMs for Fake News Generationcited by 0
2026The Persistent Vulnerability of Aligned AI Systemscited by 0
2026Minimal, Local, Causal Explanations for Jailbreak Success in Large Language Modelscited by 0
2026Exemplar Partitioning for Mechanistic Interpretabilitycited by 0
2026Residual Paving: Diagnosing the Routing Bottleneck in Selective Refusal Editingcited by 0
2026Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Modelscited by 0
2026Beyond a Single Direction: Chain-of-Thought Disrupts Simple Steering of Refusalcited by 0

50 most cited of 88 All 88 (JSON)

Similar, not cited

Close in wording, no citation either way