mlsanityData to 14 Oct 2026

Safe-SAIL: Towards a Fine-grained Safety Landscape of Large Language Models via Sparse Autoencoder Interpretation Framework

arXiv 2509.18127 ↗First posted 11 Sep 2025cs.LG cs.AI cs.CLcited by 6

On the map: Mechanistic interpretability → Steering and probing for LLM safety

Abstract (arXiv)

Sparse autoencoders (SAEs) enable interpretability research by decomposing entangled model activations into monosemantic features. However, under what circumstances SAEs derive most fine-grained latent features for safety, a low-frequency concept domain, remains unexplored. Two key challenges exist: identifying SAEs with the greatest potential for generating safety domain-specific features, and the prohibitively high cost of detailed feature explanation. In this paper, we propose Safe-SAIL, a unified framework for interpreting SAE features in safety-critical domains to advance mechanistic understanding of large language models. Safe-SAIL introduces a pre-explanation evaluation metric to efficiently identify SAEs with strong safety domain-specific interpretability, and reduces interpretation cost by 55% through a segment-level simulation strategy. Building on Safe-SAIL, we train a comprehensive suite of SAEs with human-readable explanations and systematic evaluations for 1,758 safety-related features spanning four domains: pornography, politics, violence, and terror. Using this resource, we conduct empirical analyses and provide insights on the effectiveness of Safe-SAIL for risk feature identification and how safety-critical entities and concepts are encoded across model layers. All models, explanations, and tools are publicly released in our open-source toolkit and companion product.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Steering and probing for LLM safety, one of 13 directions in Mechanistic interpretability. Builds on 15 of its 29 references that are in the map. First posted on arXiv 11 Sep 2025.

Builds on

15 of 29 references.

References · 15

  1. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  4. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  5. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  6. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
  7. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
  8. 2022A New Generation of Perspective API: Efficient Multilingual Character-level Transformerscited by 185
  9. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
  10. 2025Learning Multi-Level Features with Matryoshka Sparse Autoencoderscited by 143
  11. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 124
  12. 2024Uncovering Safety Risks of Large Language Models through Concept Activation Vectorcited by 67
3 more
  1. 2023Adversarial Attacks and Defenses in Large Language Models: Old and New Threatscited by 49
  2. 2024Adversarial Attacks on Large Language Models Using Regularized Relaxationcited by 12
  3. 2025Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoderscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse Autoencoder Features for Classifications and Transferability
  2. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet
  3. 2024Towards Inference-time Category-wise Safety Steering for Large Language Models
  4. 2024Steering Language Model Refusal with Sparse Autoencoders
  5. 2025Don't Forget It! Conditional Sparse Autoencoder Clamping Works for Unlearning
  6. 2025Safety Pretraining: Toward the Next Generation of Safe AI
  7. 2026Descriptive Collision in Sparse Autoencoder Auto-Interpretability: When One Explanation Describes Many Features
  8. 2026When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability