mlsanityData to 7 Oct 2026

SAEScientist-Bench: Can AI Agents Conduct Autonomous SAE Interpretability Research?

arXiv 2609.09113 ↗First posted 8 Sep 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment. Mechanistic interpretability tools are essential to bridge this gap, among which Sparse Autoencoders (SAEs) serve as a cornerstone by isolating interpretable features for model inspection and steering. In this paper, we introduce SAEScientist-Bench to evaluate whether AI agents can act as scientists utilizing SAE tools for autonomous mechanistic discovery. Given a target concept, an agent designs contrastive probes and navigates a Gemma Scope dictionary of 131K+ features in Gemma-2-9B-IT to discover the optimal feature, evaluated against curated expert reference features anchored on Neuronpedia across activation rank, concept selectivity on contrastive texts, and causal steering. Across 10 agent configurations and 20 tasks, frontier agents demonstrate genuine discovery capabilities and lead different evaluation dimensions, but remain well behind the expert baseline, approaching expert levels on separating target concepts from contrastive controls while lagging substantially in causal generation steering. Further analysis reveals that although agents can design contrasts to rule out spurious candidates, they frequently misinterpret experimental measurements. These results establish experimental model understanding as a measurable capability for closed-loop autonomous AI R&D. Our code is available at https://github.com/Trae1ounG/SAEScientist.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 50 of its 61 references that are in the map. First posted on arXiv 8 Sep 2026.

Builds on

50 of 61 references.

References · 50

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  4. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  5. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  6. 2024The AI Scientist: Towards Fully Automated Open-Ended Scientific Discoverycited by 1,002
  7. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  8. 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 478
  9. 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 429
  10. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  11. 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 381
  12. 2025Darwin Godel Machine: Open-Ended Evolution of Self-Improving Agentscited by 294
38 more
  1. 2025PaperBench: Evaluating AI's Ability to Replicate AI Researchcited by 286
  2. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 283
  3. 2024ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discoverycited by 208
  4. 2025AxBench: Steering LLMs? Even Simple Baselines Outperform Sparse Autoencoderscited by 199
  5. 2022Defining and Characterizing Reward Hackingcited by 184
  6. 2026Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnetcited by 175
  7. 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 148
  8. 2025SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretabilitycited by 118
  9. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 106
  10. 2026Kimi K3: Open Frontier Intelligencecited by 99
  11. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 80
  12. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  13. 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 74
  14. 2024A Multimodal Automated Interpretability Agentcited by 62
  15. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  16. 2023Neuron-Level Knowledge Attribution in Large Language Modelscited by 37
  17. 2026AIRS-Bench: a Suite of Tasks for Frontier AI Research Science Agentscited by 32
  18. 2026AuditBench: Evaluating Alignment Auditing Techniques on Models with Hidden Behaviorscited by 32
  19. 2025Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformerscited by 30
  20. 2025Bottom-up Policy Optimization: Your Language Model Policy Secretly Contains Internal Policiescited by 22
  21. 2026ResearchClawBench: A Benchmark for End-to-End Autonomous Scientific Researchcited by 13
  22. 2026RSIBench-Data: Benchmarking Data-Centric Research for Recursive Self-Improvementcited by 13
  23. 2026SoundnessBench: Can Your AI Scientist Really Tell Good Research Ideas from Bad Ones?cited by 11
  24. 2025CorrSteer: Generation-Time LLM Steering via Correlated Sparse Autoencoder Featurescited by 10
  25. 2025SAGE: An Agentic Explainer Framework for Interpreting SAE Features in Language Modelscited by 10
  26. 2026AutoResearchBench: Benchmarking AI Agents on Complex Scientific Literature Discoverycited by 10
  27. 2025Does higher interpretability imply better utility? A Pairwise Analysis on Sparse Autoencoderscited by 9
  28. 2026The Story is Not the Science: Execution-Grounded Evaluation of Mechanistic Interpretability Researchcited by 6
  29. 2025Evaluating SAE interpretability without explanationscited by 5
  30. 2025Beyond Input Activations: Identifying Influential Latents by Gradient Sparse Autoencoderscited by 3
  31. 2026Automated Interpretability and Feature Discovery in Language Models with Agentscited by 3
  32. 2026Are Sparse Autoencoder Benchmarks Reliable?cited by 3
  33. 2024SAGE: Scalable Ground Truth Evaluations for Large Sparse Autoencoderscited by 2
  34. 2025Neural Incompatibility: The Unbridgeable Gap of Cross-Scale Parametric Knowledge Transfer in Large Language Modelscited by 2
  35. 2026Steering LLMs? Actually, Sparse Autoencoders can outperform simple baselinescited by 2
  36. 2026SWE-Touch: Benchmarking Coding Agents When Users Touch the Codecited by 2
  37. 2026Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?cited by 1
  38. 2026Enhancing SAE-based Steering via Neighbor Integrated Feature Selectioncited by 1

Built on by

Not cited yet.