mlsanityData to 13 Oct 2026

Mechanistic?

arXiv 2410.09087 ↗First posted 7 Oct 2024cs.AI cs.CL cs.LGcited by 25

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

The rise of the term "mechanistic interpretability" has accompanied increasing interest in understanding neural models -- particularly language models. However, this jargon has also led to a fair amount of confusion. So, what does it mean to be "mechanistic"? We describe four uses of the term in interpretability research. The most narrow technical definition requires a claim of causality, while a broader technical definition allows for any exploration of a model's internals. However, the term also has a narrow cultural definition describing a cultural movement. To understand this semantic drift, we present a history of the NLP interpretability community and the formation of the separate, parallel "mechanistic" interpretability community. Finally, we discuss the broad cultural definition -- encompassing the entire field of interpretability -- and why the traditional NLP interpretability community has come to embrace it. We argue that the polysemy of "mechanistic" is the product of a critical divide within the interpretability community.

Key facts. Cited by 25 papers in the map (data through 13 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 120 of its 203 references that are in the map. First posted on arXiv 7 Oct 2024.

Builds on

120 of 203 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024Transcoders Find Interpretable LLM Feature Circuits17 Jun
  2. 2024Mechanistic Interpretability for AI Safety -- A Review22 Apr
  3. 2023Circuit Component Reuse Across Tasks in Transformer Language Models12 Oct
  4. 2023Progress measures for grokking via mechanistic interpretability12 Jan
  5. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretability11 Jan
  6. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 114

  1. 2017Attention Is All You Needcited by 60,106
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,304
  3. 2020Language Models are Few-Shot Learnerscited by 32,320
  4. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,055
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,539
  6. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,360
  7. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,467
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,650
  9. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,360
  10. 2017Learning Important Features Through Propagating Activation Differencescited by 1,455
  11. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,410
  12. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,391
38 more
  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,280
  2. 2014How transferable are features in deep neural networks?cited by 1,219
  3. 2018Sanity Checks for Saliency Mapscited by 1,152
  4. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,049
  5. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,009
  6. 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 999
  7. 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 992
  8. 2014Striving for Simplicity: The All Convolutional Netcited by 989
  9. 2019Attention is not Explanationcited by 922
  10. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 850
  11. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 849
  12. 2019How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddingscited by 774
  13. 2020Quantifying Attention Flow in Transformerscited by 767
  14. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 693
  15. 2019Attention is not not Explanationcited by 633
  16. 2019Designing and Interpreting Probes with Control Taskscited by 620
  17. 2020Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?cited by 594
  18. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 529
  19. 2017Interpretation of Neural Networks is Fragilecited by 443
  20. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 442
  21. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 423
  22. 2021Causal Abstractions of Neural Networkscited by 408
  23. 2019Is Attention Interpretable?cited by 388
  24. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 351
  25. 2019Revealing the Dark Secrets of BERTcited by 300
  26. 2024Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoderscited by 292
  27. 2016Rationalizing Neural Predictionscited by 281
  28. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 231
  29. 2016Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Taskscited by 216
  30. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  31. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 201
  32. 2015Visualizing and Understanding Neural Models in NLPcited by 199
  33. 2021Thinking Like Transformerscited by 182
  34. 2020Compositional Explanations of Neuronscited by 172
  35. 2017Visualizing Deep Neural Network Decisions: Prediction Difference Analysiscited by 166
  36. 2017What do Neural Machine Translation Models Learn about Morphology?cited by 166
  37. 2015Visualizing and Understanding Recurrent Networkscited by 159
  38. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 13; 2026: 12.

25 papers.

Sort:

By area:

  1. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 50
  2. 2025Language Models use Lookbacks to Track Beliefscited by 36
  3. 2026There Will Be a Scientific Theory of Deep Learningcited by 27
  4. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  5. 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 20
  6. 2025The Dead Salmons of AI Interpretabilitycited by 11
  7. 2025How Causal Abstraction Underpins Computational Explanationcited by 10
  8. 2025Adversarial Attacks Leverage Interference Between Features in Superpositioncited by 8
  9. 2025Mechanistic Interpretability Needs Philosophycited by 6
  10. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  11. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  12. 2025Aligned Probing: Relating Toxic Behavior and Model Internalscited by 5
13 more
  1. 2025VITAL: More Understandable Feature Visualization through Distribution Alignment and Relevant Information Flowcited by 3
  2. 2025ExPLAIND: Unifying Model, Data, and Training Attribution to Study Model Behaviorcited by 3
  3. 2026One Bias After Another: Mechanistic Reward Shaping and Persistent Biases in Language Reward Modelscited by 3
  4. 2026Inner Loop Inference for Pretrained Transformers: Unlocking Latent Capabilities Without Trainingcited by 2
  5. 2026How Open Must Language Models be to Enable Reliable Scientific Inference?cited by 2
  6. 2026Evaluating the Interpretability of Sparse Autoencoders with Concept Annotationscited by 2
  7. 2026Position: Explainability Research Must Prioritize Foundations over Ad-hoc Methodscited by 2
  8. 2025Constructive Circuit Amplification: Improving Math Reasoning in LLMs via Targeted Sub-Network Updatescited by 1
  9. 2026Residual Connections and the Causal Shift: Uncovering a Structural Misalignment in Transformerscited by 1
  10. 2026In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Headscited by 1
  11. 2026A Hybrid Modeling Framework for Crop Prediction Tasks via Dynamic Parameter Calibration and Multi-Task Learningcited by 0
  12. 2026MI-MIDI: Mechanistic Interpretability of Text-to-MIDI Generation Models via Probing, Lenses and Steeringcited by 0
  13. 2026Who Is Left of Whom? Tracing Spatial Evidence and Role Binding in Relative-Position Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025A Mathematical Philosophy of Explanations in Mechanistic Interpretability -- The Strange Science Part I.i
  2. 2024A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models
  3. 2025Mechanistic Indicators of Understanding in Large Language Models
  4. 2026From Mechanistic to Compositional Interpretability
  5. 2024The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms
  6. 2025Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research
  7. 2025Open Problems in Mechanistic Interpretability
  8. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models