mlsanityData to 7 Oct 2026

Steering Language Model Refusal with Sparse Autoencoders

arXiv 2411.11296 ↗First posted 18 Nov 2024cs.LGcited by 89

Not on the map. Why?

Cited by 32 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Responsible deployment of language models requires mechanisms for refusing unsafe prompts while preserving model performance. While most approaches modify model weights through additional training, we explore an alternative: steering model activations at inference time via amplifying sparse autoencoder (SAE) features that mediate refusal. This work uncovers a fundamental tension between SAE steering-based safety improvements and general model capabilities. While feature steering successfully improves robustness against both single-turn and challenging multi-turn jailbreak attempts, we discover that this comes at a previously underexplored cost -- systematic degradation of performance across multiple benchmark tasks, even on safe inputs with no apparent connection to refusal behavior. This suggests that features mediating refusal may be more deeply entangled with general language model capabilities than previously understood. Our findings reveal important open questions about the nature of safety-relevant features in language models and the feasibility of isolating them for targeted intervention. While SAE-based steering shows promise as a flexible approach to enhancing language model safety, our results highlight the critical need to understand and address the mechanisms behind these capability tradeoffs before such techniques can be practically deployed.

Key facts. Cited by 89 papers in the map (data through 7 Oct 2026). Builds on 74 of its 91 references that are in the map. First posted on arXiv 18 Nov 2024.

Builds on

74 of 91 references.

References · 74

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  7. 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,883
  8. 2023Jailbroken: How Does LLM Safety Training Fail?cited by 1,565
  9. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  10. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,330
  11. 2023Jailbreaking Black Box Large Language Models in Twenty Queriescited by 1,326
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
38 more
  1. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,239
  2. 2024The FineWeb Datasets: Decanting the Web for the Finest Text Data at Scalecited by 1,104
  3. 2022Editing Models with Task Arithmeticcited by 1,076
  4. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,009
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 960
  6. 2023Steering Language Models With Activation Engineeringcited by 824
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 702
  8. 2017Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problemscited by 642
  9. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 598
  10. 2024Scaling and evaluating sparse autoencoderscited by 530
  11. 2022Improving alignment of dialogue agents via targeted human judgementscited by 527
  12. 2024The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearningcited by 488
  13. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  14. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  15. 2024Safety Alignment Should Be Made More Than Just a Few Tokens Deepcited by 450
  16. 2023LMSYS-Chat-1M: A Large-Scale Real-World LLM Conversation Datasetcited by 449
  17. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 434
  18. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 419
  19. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  20. 2024AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agentscited by 406
  21. 2024Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attackcited by 364
  22. 2023Are aligned neural networks adversarially aligned?cited by 305
  23. 2023Shadow Alignment: The Ease of Subverting Safely-Aligned Language Modelscited by 284
  24. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  25. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  26. 2023In-context Vectors: Making In Context Learning More Effective and Controllable Through Latent Space Steeringcited by 240
  27. 2022Extracting Latent Steering Vectors from Pretrained Language Modelscited by 217
  28. 2024Jailbreak Attacks and Defenses Against Large Language Models: A Surveycited by 215
  29. 2013k-Sparse Autoencoderscited by 209
  30. 2024WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Modelscited by 206
  31. 2024Programming Refusal with Conditional Activation Steeringcited by 191
  32. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  33. 2024Improving Instruction-Following in Language Models through Activation Steeringcited by 149
  34. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  35. 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 138
  36. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  37. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  38. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106

50 most cited of 74 All 74 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 34; 2026: 54.

89 papers.

Sort:

By area:

  1. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  2. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 104
  3. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraintcited by 45
  4. 2025Enhancing Automated Interpretability with Output-Centric Feature Descriptionscited by 39
  5. 2025Safety is Not Only About Refusal: Reasoning-Enhanced Fine-tuning for Interpretable LLM Safetycited by 38
  6. 2026There Is More to Refusal in Large Language Models than a Single Directioncited by 22
  7. 2025SAEs Can Improve Unlearning: Dynamic Sparse Autoencoder Guardrails for Precision Unlearning in LLMscited by 21
  8. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  9. 2025Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusalcited by 17
  10. 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
  11. 2025SAUCE: Selective Concept Unlearning in Vision-Language Models with Sparse Autoencoderscited by 12
  12. 2025Incorporating Hierarchical Semantics in Sparse Autoencoder Architecturescited by 12
38 more
  1. 2024Safeguard Fine-Tuned LLMs Through Pre- and Post-Tuning Model Mergingcited by 11
  2. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 11
  3. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Featurescited by 10
  4. 2025Jacobian Sparse Autoencoders: Sparsify Computations, Not Just Activationscited by 9
  5. 2025Using Mechanistic Interpretability to Craft Adversarial Attacks against Large Language Modelscited by 9
  6. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  7. 2026Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoningcited by 9
  8. 2025Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLMcited by 8
  9. 2025Model Directions, Not Words: Mechanistic Topic Models Using Sparse Autoencoderscited by 7
  10. 2025Interpreting Large Text-to-Image Diffusion Models with Dictionary Learningcited by 6
  11. 2026From Data to Behavior: Predicting Unintended Model Behaviors Before Trainingcited by 6
  12. 2026Among Us: Measuring and Mitigating Malicious Contributions in Model Collaboration Systemscited by 6
  13. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 6
  14. 2026Estimating Tail Risks in Language Model Output Distributionscited by 6
  15. 2025Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injectioncited by 5
  16. 2025STAC: When Innocent Tools Form Dangerous Chains for LLM Agentscited by 5
  17. 2025Resa: Transparent Reasoning Models via SAEscited by 4
  18. 2025AlignSAE: Concept-Aligned Sparse Autoencoderscited by 4
  19. 2025Depth-Wise Activation Steering for Honest Language Modelscited by 4
  20. 2026Sparse Autoencoders are Capable LLM Jailbreak Mitigatorscited by 4
  21. 2026Suppression or Deletion: A Restoration-Based Representation-Level Analysis of Machine Unlearningcited by 4
  22. 2026Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selectioncited by 4
  23. 2026Perplexity Can Miss SAE Feature Damage Under Quantizationcited by 4
  24. 2025Toward a Flexible Framework for Linear Representation Hypothesis Using Maximum Likelihood Estimationcited by 3
  25. 2025Ensembling Sparse Autoencoderscited by 3
  26. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steeringcited by 3
  27. 2025Generalization or Memorization: Dynamic Decoding for Mode Steeringcited by 3
  28. 2026Interpreting and Steering State-Space Models via Activation Subspace Bottleneckscited by 3
  29. 2025Patterns and Mechanisms of Contrastive Activation Engineeringcited by 2
  30. 2025Insights into a radiology-specialised multimodal large language model with sparse autoencoderscited by 2
  31. 2025ConceptGuard: Neuro-Symbolic Safety Guardrails via Sparse Interpretable Jailbreak Conceptscited by 2
  32. 2025Interpretable Safety Alignment via SAE-Constructed Low-Rank Subspace Adaptationcited by 2
  33. 2026Exploring Diverse Generation Paths via Inference-time Stiefel Activation Steeringcited by 2
  34. 2026Mind the Performance Gap: Capability-Behavior Trade-offs in Feature Steeringcited by 2
  35. 2026Don't Lose Focus: Activation Steering via Key-Orthogonal Projectionscited by 2
  36. 2026LLM Advertisement based on Neuron Auctionscited by 2
  37. 2026Toward Identifiable Sparse Autoencoderscited by 2
  38. 2025Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generationcited by 1

50 most cited of 89 All 89 (JSON)