Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework
On the map: Mechanistic interpretability → Steering and probing for LLM safety
Abstract (arXiv)
Sparse autoencoders (SAEs) enable interpretability research by decomposing entangled model activations into monosemantic features. However, under what circumstances SAEs derive most fine-grained latent features for safety, a low-frequency concept domain, remains unexplored. Two key challenges exist: identifying SAEs with the greatest potential for generating safety domain-specific features, and the prohibitively high cost of detailed feature explanation. In this paper, we propose Safe-SAIL, a unified framework for interpreting SAE features in safety-critical domains to advance mechanistic understanding of large language models. Safe-SAIL introduces a pre-explanation evaluation metric to efficiently identify SAEs with strong safety domain-specific interpretability, and reduces interpretation cost by 55% through a segment-level simulation strategy. Building on Safe-SAIL, we train a comprehensive suite of SAEs with human-readable explanations and systematic evaluations for 1,758 safety-related features spanning four domains: pornography, politics, violence, and terror. Using this resource, we conduct empirical analyses and provide insights on the effectiveness of Safe-SAIL for risk feature identification and how safety-critical entities and concepts are encoded across model layers. All models, explanations, and tools are publicly released in our open-source toolkit and companion product.
Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 29 references that are in the map. First posted on arXiv 11 Sep 2025.
Builds on
15 of 29 references.
References · 15
- 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
- 2023Bias and Fairness in Large Language Models: A Surveycited by 639
- 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
- 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
- 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
- 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
- 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
- 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
- 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
- 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
Built on by
Citing papers per year: 2026: 6.
6 papers.
- 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Modelscited by 27
- 2026NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labelscited by 4
- 2026AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attackscited by 3
- 2026Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Modelscited by 3
- 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMscited by 0
- 2026Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utilitycited by 0
- 2026Sparse Autoencoders for Mechanistic Interpretability in NLP: A Critical Survey of Recoverability, Interpretability, and Utility14 Oct 2026
- 2026When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs30 Aug 2026
- 2026Why Does Reinforcement Learning Generalize? A Feature-Level Mechanistic Study of Post-Training in Large Language Models27 Apr 2026
- 2026AISA: Awakening Intrinsic Safety Awareness in Large Language Models against Jailbreak Attacks14 Feb 2026
- 2026NExT-Guard: Training-Free Streaming Safeguard without Token-Level Labels10 Feb 2026
- 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models20 Jan 2026
Similar, not cited
Close in wording, no citation either way
- 2025Sparse Autoencoder Features for Classifications and Transferability
- 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
- 2024Towards Inference-time Category-wise Safety Steering for Large Language Models
- 2024Steering Language Model Refusal with Sparse Autoencoders
- 2025Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning
- 2025Safety Pretraining: Toward the Next Generation of Safe AI
- 2026Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
- 2026When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability