mlsanityData to 14 Oct 2026

SAE-StatSteer: Statistical Consensus Feature Selection for Optimization-Free Activation Steering of Large Language Models

arXiv 2607.19364 ↗First posted 5 Jun 2026cs.AI cs.CLcited by 0

On the map: Mechanistic interpretability → Activation steering methods

Abstract (arXiv)

Activation steering adds a residual-stream direction at inference time, providing lightweight behavioral control without fine-tuning. Sparse autoencoders (SAEs) can make such interventions auditable by decomposing dense activations into an approximately monosemantic feature basis. We introduce SAE-StatSteer, a transparent, optimization-free pipeline. It first filters features through six reliability conditions, then ranks the survivors by an unweighted Borda consensus over three statistics, an F-test, KSG mutual information, and Cohen's d, and finally combines the selected SAE decoder rows using Cohen's-d weights. We evaluate three Gemma-family models across four behavioral domains against seven dense or SAE-based baselines. Our quality-conditioned protocol requires attribute movement while preserving relevance, richness, and coherence. Raw success systematically overstates usable control because strong shifts often degrade generation quality, and effective steering is not governed by a universal layer or strength. SAE-StatSteer remains competitive with optimization-based methods while exposing every selection and weighting decision for audit. These results motivate reporting quality-conditioned success alongside raw behavioral shift. Our code and data are available at https://github.com/Oshayer-Siddique/LLM-Steering-Using-SAE.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Activation steering methods, one of 13 directions in Mechanistic interpretability. Builds on 22 of its 33 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

22 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025A Unified Understanding and Evaluation of Steering Methods4 Feb
  2. 2023Steering Llama 2 via Contrastive Activation Addition9 Dec
  3. 2023Steering Language Models With Activation Engineering20 Aug

Other references · 19

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  3. 2024A Survey on LLM-as-a-Judgecited by 1,423
  4. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  7. 2023Large Language Modelscited by 780
  8. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  9. 2020Aligning AI With Shared Human Valuescited by 723
  10. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  11. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
  12. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 300
7 more
  1. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
  2. 2025Are Sparse Autoencoders Useful? A Case Study in Sparse Probingcited by 115
  3. 2024Improving Steering Vectors by Targeting Sparse Autoencoder Featurescited by 92
  4. 2024On the Relationship between Truth and Political Bias in Language Modelscited by 31
  5. 2025SAE-SSV: Supervised Steering in Sparse Representation Spaces for Reliable Control of Language Modelscited by 29
  6. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 15
  7. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 10

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Steering Large Language Model Activations in Sparse Spaces
  2. 2026Enhancing SAE-based Steering via Neighbor Integrated Feature Selection
  3. 2026A Coin Flip Per Token: Bernoulli Sparse Steering of Large Language Models
  4. 2025A Comparative Analysis of Sparse Autoencoder and Activation Difference in Language Model Steering
  5. 2026Pre-Intervention Prediction of Sparse Autoencoder Steering Side Effects
  6. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Features
  7. 2025Interpretable Steering of Large Language Models with Feature Guided Activation Additions
  8. 2026Strengthening Target-Language Features: SAE-Based Steering for Multilingual Inference