mlsanityData to 7 Oct 2026

Open Problems in Mechanistic Interpretability

arXiv 2501.16496 ↗First posted 27 Jan 2025cs.LGcited by 173

Not on the map. Why?

Abstract (arXiv)

Mechanistic interpretability aims to understand the computational mechanisms underlying neural networks' capabilities in order to accomplish concrete scientific and engineering goals. Progress in this field thus promises to provide greater assurance over AI system behavior and shed light on exciting scientific questions about the nature of intelligence. Despite recent progress toward these goals, there are many open problems in the field that require solutions before many scientific and practical benefits can be realized: Our methods require both conceptual and practical improvements to reveal deeper insights; we must figure out how best to apply our methods in pursuit of specific goals; and the field must grapple with socio-technical challenges that influence and are influenced by our work. This forward-facing review discusses the current frontier of mechanistic interpretability and the open problems that the field may benefit from prioritizing.

Key facts. Cited by 173 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Jan 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 81; 2026: 92.

173 papers.

Sort:

By area:

  1. 2025Agentic Large Language Models, a surveycited by 105
  2. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  3. 2025The Non-Linear Representation Dilemma: Is Causal Abstraction Enough for Mechanistic Interpretability?cited by 34
  4. 2025Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuningcited by 31
  5. 2025Training Language Models to Explain Their Own Computationscited by 27
  6. 2026Do Sparse Autoencoders Capture Concept Manifolds?cited by 25
  7. 2025How Visual Representations Map to Language Feature Space in Multimodal LLMscited by 24
  8. 2025Beyond Prompt Engineering: Robust Behavior Control in LLMs via Steering Target Atomscited by 23
  9. 2025How Do LLMs Acquire New Knowledge? A Knowledge Circuits Perspective on Continual Pre-Trainingcited by 22
  10. 2026There Will Be a Scientific Theory of Deep Learningcited by 21
  11. 2025Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesiscited by 19
  12. 2025The Rogue Scalpel: Activation Steering Compromises LLM Safetycited by 18
38 more
  1. 2026Language Model Circuits Are Sparse in the Neuron Basiscited by 18
  2. 2025Transferring Linear Features Across Language Models With Model Stitchingcited by 17
  3. 2025Position: Use Sparse Autoencoders to Discover Unknownscited by 17
  4. 2025ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMscited by 16
  5. 2025Risk Analysis Techniques for Governed LLM-based Multi-Agent Systemscited by 16
  6. 2026Qwen-Scope: Turning Sparse Features into Development Tools for Large Language Modelscited by 15
  7. 2025Does Time Have Its Place? Temporal Heads: Where Language Models Recall Time-specific Informationcited by 14
  8. 2025Because we have LLMs, we Can and Should Pursue Agentic Interpretabilitycited by 14
  9. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  10. 2025Alternating Gradient Flows: A Theory of Feature Learning in Two-layer Neural Networkscited by 11
  11. 2026Do Sparse Autoencoders Identify Reasoning Features in Language Models?cited by 11
  12. 2026Coding Agents are Effective Long-Context Processorscited by 11
  13. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 10
  14. 2026Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guaranteescited by 10
  15. 2025Capturing Polysemanticity with PRISM: A Multi-Concept Feature Description Frameworkcited by 9
  16. 2025The Singapore Consensus on Global AI Safety Research Prioritiescited by 9
  17. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 9
  18. 2026The Information Geometry of Softmax: Probing and Steeringcited by 9
  19. 2025Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Modelscited by 8
  20. 2025CytoSAE: Interpretable Cell Embeddings for Hematologycited by 8
  21. 2025Distributional AGI Safetycited by 8
  22. 2026When Agents Say One Thing and Do Another: Validating Elicited Beliefs from LLMscited by 8
  23. 2026All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMscited by 8
  24. 2025Watch the Weights: Unsupervised monitoring and control of fine-tuned LLMscited by 7
  25. 2026Sequential Group Composition: A Window into the Mechanics of Deep Learningcited by 7
  26. 2026The Expert Strikes Back: Interpreting Mixture-of-Experts Language Models at Expert Levelcited by 7
  27. 2025On the Fundamental Impossibility of Hallucination Control in Large Language Modelscited by 6
  28. 2025Mechanistic Interpretability Needs Philosophycited by 6
  29. 2025Mechanistic Indicators of Understanding in Large Language Modelscited by 6
  30. 2025Limits of Generalization in RLVR: Two Case Studies in Mathematical Reasoningcited by 6
  31. 2026Establishing Construct Validity in LLM Capability Benchmarks Requires Nomological Networkscited by 6
  32. 2026Catching rationalization in the act: detecting motivated reasoning before and after CoT via activation probingcited by 6
  33. 2026Position: Anthropomorphic Misalignment Research Needs Stronger Evidencecited by 6
  34. 2025Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoderscited by 5
  35. 2025On the creation of narrow AI: hierarchy and nonlocality of neural network skillscited by 4
  36. 2025Probing Neural Topology of Large Language Modelscited by 4
  37. 2025From Concepts to Components: Concept-Agnostic Attention Module Discovery in Transformerscited by 4
  38. 2025Feature Identification via the Empirical NTKcited by 4

50 most cited of 173 All 173 (JSON)