Mechanist: AI as a Scientific Instrument for Discovering the Mechanisms of Intelligence
Not on the map. Why?
Abstract (arXiv)
AI models are increasingly used in scientific discovery and human decision-making. Yet how AI models work and what risks they pose remain poorly understood. As AI development becomes faster and more automated, research on the mechanisms underlying AI remains largely manual. To bridge this gap, we introduce Mechanist, an agentic system that uses AI as a scientific instrument for the autonomous discovery of mechanisms underlying AI. To ground novel mechanism hypotheses, we construct a scientific knowledge graph of 13,000 studies on AI mechanisms, alongside a multidisciplinary database of 43 million papers spanning 26 fields. For reliable experiment execution, we curate a library of 32 foundational methods for mechanism analysis. Compared with Claude Code and existing AI-scientist systems, Mechanist generates higher-quality mechanism hypotheses and executes experiments more reliably. Across four case studies, Mechanist autonomously discovers new model behaviors and their underlying mechanisms, and translates these discoveries into mechanism-guided interventions and interdisciplinary design. Specifically, Mechanist first uncovers a counterintuitive safety risk in scientific laboratories, showing that unsafe traits can transfer to fine-tuned student models through apparently safe training data and emerge across modalities. Mechanist then develops a mechanism theory of belief, revealing how models represent world knowledge, form beliefs, infer the beliefs of others, and how these mechanisms emerge during pretraining. Building on this theory, Mechanist develops targeted interventions that improve model performance across diverse scenarios. Finally, Mechanist can also advance interdisciplinary discovery through mechanistic design, providing an alternative to the computationally intensive generate-and-rerank paradigm.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 72 of its 101 references that are in the map. First posted on arXiv 12 Aug 2026.
Builds on
72 of 101 references.
References · 72
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
- 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
- 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
- 2022Editing Models with Task Arithmeticcited by 1,120
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
- 2024Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Trainingcited by 529
- 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
- 2024MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineeringcited by 469
38 more
- 2024AFlow: Automating Agentic Workflow Generationcited by 432
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
- 2025Accelerating scientific discovery with Co-Scientistcited by 401
- 2024Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2cited by 394
- 2024Alignment faking in large language modelscited by 360
- 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 285
- 2024LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinationscited by 244
- 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 209
- 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judgecited by 204
- 2024Transcoders Find Interpretable LLM Feature Circuitscited by 200
- 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
- 2025Emergent Misalignment: Narrow finetuning can produce broadly misaligned LLMscited by 172
- 2024A Comprehensive Study of Knowledge Editing for Large Language Modelscited by 163
- 2024A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoderscited by 160
- 2026Towards End-to-End Automation of AI Researchcited by 159
- 2024Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanismscited by 149
- 2024Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoderscited by 143
- 2022From Attribution Maps to Human-Understandable Explanations through Concept Relevance Propagationcited by 141
- 2024Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Modelscited by 126
- 2021FastSHAP: Real-Time Shapley Value Estimationcited by 125
- 2025Multi-Agent Design: Optimizing Agents with Better Prompts and Topologiescited by 113
- 2024Self-Discover: Large Language Models Self-Compose Reasoning Structurescited by 110
- 2025Multiagent Finetuning: Self Improvement with Diverse Reasoning Chainscited by 95
- 2024TruthX: Alleviating Hallucinations by Editing Large Language Models in Truthful Spacecited by 91
- 2025A Self-Improving Coding Agentcited by 79
- 2025SAEs Are Good for Steering -- If You Select the Right Featurescited by 78
- 2024InterPLM: Discovering Interpretable Features in Protein Language Models via Sparse Autoencoderscited by 66
- 2025ML-Master: Towards AI-for-AI via Integration of Exploration and Reasoningcited by 61
- 2025Massive Values in Self-Attention Modules are the Key to Contextual Knowledge Understandingcited by 47
- 2025Inoculation Prompting: Eliciting traits from LLMs during training can suppress them at test-timecited by 43
- 2025Inoculation Prompting: Instructing LLMs to misbehave at train-time improves test-time alignmentcited by 43
- 2026The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlookcited by 43
- 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 34
- 2025Towards Understanding Subliminal Learning: When and How Hidden Biases Transfercited by 34
- 2024What Happened in LLMs Layers when Trained for Fast vs. Slow Thinking: A Gradient Perspectivecited by 29
- 2025Mechanistic understanding and validation of large AI models with SemanticLenscited by 29
- 2023Neuron to Graph: Interpreting Language Model Neurons at Scalecited by 27
- 2022Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Modelcited by 25
50 most cited of 72 All 72 (JSON)
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026MechBench: Can AI Scientific Agents Discover Mechanisms Beyond Phenomenal Laws?
- 2026From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery
- 2025Agentic AI for Scientific Discovery: A Survey of Progress, Challenges, and Future Directions
- 2025Virtuous Machines: Towards Artificial General Science
- 2025Scaling Laws in Scientific Discovery with AI and Robot Scientists
- 2025From AI for Science to Agentic Science: A Survey on Autonomous Scientific Discovery
- 2026Measuring Biological Capabilities and Risks of AI Agents
- 2025Superintelligent Agents Pose Catastrophic Risks: Can Scientist AI Offer a Safer Path?