mlsanityData to 14 Oct 2026

Graph-Regularized Sparse Autoencoders for LLM Safety Steering

arXiv 2512.06655 ↗First posted 7 Dec 2025cs.LG cs.AIcited by 1

On the map: LLM safety and security → Safety alignment, refusal and guardrails

also in 1 more
Abstract (arXiv)

Sparse autoencoders (SAEs) are increasingly used to extract activation directions for inference-time steering, but their standard sparsity objective treats latent features as independent. This prior can be poorly matched to high-level safety behaviors, where refusal and harmful compliance appear to depend on distributed structure in activation space. We introduce Graph-Regularized Sparse Autoencoders (GSAE), a dictionary-learning method that learns safety-steering directions by smoothing SAE decoder vectors over a neuron co-activation graph and applying the resulting direction bank through a two-gate runtime controller. Empirically, GSAE improves selective refusal across JailbreakBench, HarmBench, and XSTest, increasing harmful-request refusal while keeping benign-prompt refusals low. On Llama-3-8B, replacing the standard SAE with GSAE in an otherwise identical pipeline improves Deltas by 20.1 points on JailbreakBench and 16.8 points on HarmBench. GSAE outperforms activation-steering baselines and black-box guardrails, preserves benign-task performance, generalizes across Llama-3, Mistral, Qwen 2.5, and Phi-4, and remains strong under black-box and gray-box jailbreak attacks.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Safety alignment, refusal and guardrails, one of 12 directions in LLM safety and security. Builds on 45 of its 56 references that are in the map. First posted on arXiv 7 Dec 2025.

Builds on

45 of 56 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence24 Feb
  2. 2025SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals3 Feb
  3. 2024Refusal in Language Models Is Mediated by a Single Direction17 Jun
  4. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vector18 Apr

Other references · 41

  1. 2017Attention Is All You Needcited by 60,361
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2023Qwen Technical Reportcited by 3,344
  6. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 3,104
  7. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  11. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  12. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
29 more
  1. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  2. 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 976
  3. 2023Steering Language Models With Activation Engineeringcited by 922
  4. 2023AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Modelscited by 785
  5. 2024Phi-4 Technical Reportcited by 711
  6. 2023Tree of Attacks: Jailbreaking Black-Box LLMs Automaticallycited by 672
  7. 2024JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Modelscited by 536
  8. 2024Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attackscited by 486
  9. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 212
  10. 2024Programming Refusal with Conditional Activation Steeringcited by 206
  11. 2024How Alignment and Jailbreak Work: Explain LLM Safety through Intermediate Hidden Statescited by 98
  12. 2024Steering Language Model Refusal with Sparse Autoencoderscited by 95
  13. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  14. 2024Gradient Cuff: Detecting Jailbreak Attacks on Large Language Models by Exploring Refusal Loss Landscapescited by 74
  15. 2024The Geometry of Concepts: Sparse Autoencoder Feature Structurecited by 52
  16. 2025Understanding Refusal in Language Models with Sparse Autoencoderscited by 29
  17. 2025Route Sparse Autoencoder to Interpret Large Language Modelscited by 26
  18. 2025COSMIC: Generalized Refusal Direction Identification in LLM Activationscited by 19
  19. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 19
  20. 2025Model Unlearning via Sparse Autoencoder Subspace Guided Projectionscited by 18
  21. 2025Interpretable Reward Model via Sparse Autoencodercited by 18
  22. 2025Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEscited by 14
  23. 2025Refusal Behavior in Large Language Models: A Nonlinear Perspectivecited by 13
  24. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 10
  25. 2025Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Samplingcited by 9
  26. 2025Denoising Concept Vectors with Sparse Autoencoders for Improved Language Model Steeringcited by 9
  27. 2025PrivacyScalpel: Enhancing LLM Privacy via Interpretable Feature Intervention with Sparse Autoencoderscited by 6
  28. 2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignmentcited by 4
  29. 2023Joint Graph Learning and Model Fitting in Laplacian Regularized Stratified Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026REINS: Refusal-Enhanced Inhibitory Steering with Sparse Autoencoder Features
  2. 2025Feature-Guided SAE Steering for Refusal-Rate Control using Contrasting Prompts
  3. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safety
  4. 2026When Are Sparse Feature Interventions Actually Localized? Matched Evaluation for SAE-Based Safety Control
  5. 2025Guiding Giants: Lightweight Controllers for Weighted Activation Steering in LLMs
  6. 2025AlphaSteer: Learning Refusal Steering with Principled Null-Space Constraint
  7. 2025Steering Large Language Model Activations in Sparse Spaces
  8. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atoms