mlsanityData to 14 Oct 2026

Enhancing SAE-based Steering via Neighbor Integrated Feature Selection

arXiv 2608.28806 ↗First posted 28 Aug 2026cs.AIcited by 1

On the map: Mechanistic interpretability → SAE features and their applications

Abstract (arXiv)

Sparse autoencoders (SAEs) disentangle model activations into interpretable features and are widely used for steering large language models. Most existing SAE-based steering methods select features by applying a top- filter based on statistical scores, assuming that higher-scoring features yield stronger steering effects. In this paper, we show that this assumption is often invalid, leading to suboptimal feature selection. Our analysis reveals that effective steering features may be distributed among representationally adjacent, semantically similar groups induced by feature splitting in SAEs. Within such groups, features may exhibit disparate statistical scores despite having comparable steering influence, causing score-based selection to overlook important features. Based on these observations, we propose textscNeighbor Integrated Feature Selection (textscNIFS), a plug-and-play strategy that leverages representation similarity to improve feature selection for steering. We evaluate textscNIFS across multiple SAE-based steering methods and tasks, and demonstrate consistent performance gains over conventional top-k selection.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in SAE features and their applications, one of 13 directions in Mechanistic interpretability. Builds on 22 of its 32 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

22 of 32 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoders4 Oct
  2. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features18 Aug
  3. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Models22 May
  4. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability12 Mar
  5. 2025SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models17 Feb
  6. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Features4 Nov
  7. 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders27 Oct
  8. 2024Steering Knowledge Selection Behaviours in LLMs via SAE-Based Representation Engineering21 Oct
  9. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders22 Sep
  10. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders19 Jul
  11. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Models15 Sep

Other references · 11

  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  2. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  3. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  4. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  5. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  6. 2021Entity-Based Knowledge Conflicts in Question Answeringcited by 356
  7. 2023Linear Representations of Sentiment in Large Language Modelscited by 163
  8. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 68
  9. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 24
  10. 2023Efficient fair PCA for fair representation learningcited by 23
  11. 2026DLM-Scope: Mechanistic Interpretability of Diffusion Language Models via Sparse Autoencoderscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?cited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models
  2. 2025SAEs Are Good for Steering -- If You Select the Right Features
  3. 2026Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselines
  4. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinement
  5. 2024Can sparse autoencoders be used to decompose and interpret steering vectors?
  6. 2026Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference
  7. 2026Multilingual Steering by Design: Multilingual Sparse Autoencoders and Principled Layer Selection
  8. 2026Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects