mlsanityData to 14 Oct 2026

Refusal Before Decoding: Detecting and Exploiting Refusal Signals in Intermediate LLM Activations

arXiv 2605.28553 ↗First posted 27 May 2026cs.AI cs.CRcited by 1

On the map: Mechanistic interpretability → Steering and probing for LLM safety

Abstract (arXiv)

In this paper, we investigate whether refusal behavior can be predicted from LLM intermediate activations before decoding using linear probes trained on residual stream activations at each transformer block. We find that refusal is linearly decodable well before the final layer, indicating that safety-relevant behavior is represented in intermediate activations before output generation. To test whether this signal is actionable, we introduce Mechanistic AutoDAN, a probe-guided variant of AutoDAN that replaces full-model fitness evaluation with partial forward passes and probe-based scoring inside a genetic prompt search loop. Across the evaluated models, our method achieves attack success rates competitive with vanilla AutoDAN while reducing per-iteration search time by up to 72%, and probe-guided prompts match or exceed AutoDAN's cross-model transfer in several configurations. We further find that the usefulness of probe guidance increases with model scale. Our results suggest that refusal-relevant information is decodable from intermediate activations and can serve as an effective search signal in an AutoDAN-style discrete jailbreak optimization loop, especially for larger and more robust models.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 21 of its 53 references that are in the map. First posted on arXiv 27 May 2026.

Builds on

21 of 53 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Models8 Mar
  2. 2024Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective12 Jan

Other references · 19

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2017Deep reinforcement learning from human preferencescited by 3,912
  3. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  4. 2020Shortcut Learning in Deep Neural Networkscited by 1,969
  5. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  6. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  9. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  10. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  11. 2024SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusalcited by 240
  12. 2023Red-Teaming Large Language Models using Chain of Utterances for Safety-Alignmentcited by 223
7 more
  1. 2022On Second Thought, Let's Not Think Step by Step! Bias and Toxicity in Zero-Shot Reasoningcited by 194
  2. 2024Language Models are Homer Simpson! Safety Re-Alignment of Fine-tuned Language Models through Task Arithmeticcited by 112
  3. 2025A Survey on Sparse Autoencoders: Interpreting the Internal Mechanisms of Large Language Modelscited by 76
  4. 2025JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulationcited by 54
  5. 2025NeuroStrike: Neuron-Level Attacks on Aligned LLMscited by 22
  6. 2025Refusal Behavior in Large Language Models: A Nonlinear Perspectivecited by 13
  7. 2025LatentBreak: Jailbreaking Large Language Models through Latent Space Feedbackcited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026From Probe Scores to Alarm Policies: Operational Validity of Activation Monitors for Language-Model Agentscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Efficient Refusal Ablation in LLM through Optimal Transport
  2. 2026Activation Surgery: Jailbreaking White-box LLMs without Touching the Prompt
  3. 2026Refusal-Gated Decoding: Preserving Refusal Behavior Under High-Temperature Sampling
  4. 2026Decoy Direction Optimization: A Post-Hoc Defense Against LLM Abliteration
  5. 2025EVOREFUSE: Evolutionary Prompt Optimization for Evaluation and Mitigation of LLM Over-Refusal to Pseudo-Malicious Instructions
  6. 2026CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoders
  7. 2026Universal Refusal Circuits Across LLMs: Cross-Model Transfer via Trajectory Replay and Concept-Basis Reconstruction
  8. 2026Step-Wise Refusal Dynamics in Autoregressive and Diffusion Language Models