mlsanityData to 14 Oct 2026

Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretability

arXiv 2305.08746 ↗First posted 4 May 2023cs.NE cond-mat.dis-nn cs.AIcited by 38

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

We introduce Brain-Inspired Modular Training (BIMT), a method for making neural networks more modular and interpretable. Inspired by brains, BIMT embeds neurons in a geometric space and augments the loss function with a cost proportional to the length of each neuron connection. We demonstrate that BIMT discovers useful modular neural networks for many simple tasks, revealing compositional structures in symbolic formulas, interpretable decision boundaries and features for classification, and mathematical structure in algorithmic datasets. The ability to directly see modules with the naked eye can complement current mechanistic interpretability strategies such as probes, interventions or staring at all weights.

Key facts. Cited by 38 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 23 of its 36 references that are in the map. First posted on arXiv 4 May 2023.

Builds on

23 of 36 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operations6 Feb
  2. 2023Progress measures for grokking via mechanistic interpretability12 Jan
  3. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small1 Nov

Other references · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2022Toy Models of Superpositioncited by 971
  5. 2022In-context Learning and Induction Headscited by 964
  6. 2022What learning algorithm is in-context learning? Investigations with linear modelscited by 684
  7. 2022Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasetscited by 629
  8. 2020What is the State of Neural Network Pruning?cited by 583
  9. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  10. 2020Discovering Symbolic Models from Deep Learning with Inductive Biasescited by 344
  11. 2022Towards Understanding Grokking: An Effective Theory of Representation Learningcited by 230
  12. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
8 more
  1. 2022Omnigrok: Grokking Beyond Algorithmic Datacited by 146
  2. 2020AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularitycited by 144
  3. 2023The Quantization Model of Neural Scalingcited by 138
  4. 2020Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Maskscited by 98
  5. 2023Modular Deep Learningcited by 83
  6. 2018Modular Networks: Learning to Decompose Neural Computationcited by 62
  7. 2021Clusterability in Neural Networkscited by 24
  8. 2015The evolutionary origins of hierarchycited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 13; 2025: 8; 2026: 7.

38 papers.

Sort:

By area:

  1. 2024KAN: Kolmogorov-Arnold Networkscited by 649
  2. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  3. 2023Challenges and Applications of Large Language Modelscited by 254
  4. 2023Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directionscited by 125
  5. 2024KAN 2.0: Kolmogorov-Arnold Networks Meet Sciencecited by 118
  6. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 51
  7. 2023Codebook Features: Sparse and Discrete Interpretability for Neural Networkscited by 48
  8. 2025Mixture of Experts Made Intrinsically Interpretablecited by 24
  9. 2023Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanicscited by 16
  10. 2024Using Degeneracy in the Loss Landscape for Mechanistic Interpretabilitycited by 13
  11. 2023Growing Brains: Co-emergence of Anatomical and Functional Modularity in Recurrent Neural Networkscited by 7
  12. 2025Low-Rank Adapting Models for Sparse Autoencoderscited by 7
26 more
  1. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  2. 2023A Neural Scaling Law from Lottery Ticket Ensemblingcited by 5
  3. 2025Machine Learning meets Algebraic Combinatorics: A Suite of Datasets Capturing Research-level Conjecturing Ability in Pure Mathematicscited by 5
  4. 2023The semantic landscape paradigm for neural networkscited by 4
  5. 2023Neural Network Pruning by Gradient Descentcited by 4
  6. 2024Efficient Model Compression Techniques with FishLegcited by 4
  7. 2025Harmonic Loss Trains Interpretable AI Modelscited by 4
  8. 2023Generating Interpretable Networks using Hypernetworkscited by 3
  9. 2024Quantum Algorithms for Compositional Text Processingcited by 3
  10. 2026The Dual-Stream Transformer: Channelized Architecture for Interpretable Language Modelingcited by 3
  11. 2023SPADE: Sparsity-Guided Debugging for Deep Neural Networkscited by 2
  12. 2024Hyperdimensional computing: a fast, robust and interpretable paradigm for biological datacited by 2
  13. 2024Machines and Mathematical Mutations: Using GNNs to Characterize Quiver Mutation Classescited by 2
  14. 2026Modularity is the Bedrock of Natural and Artificial Intelligencecited by 2
  15. 2024The Garden of Forking Paths: Observing Dynamic Parameters Distribution in Large Language Modelscited by 1
  16. 2024NeuroPrune: A Neuro-inspired Topological Sparse Training Algorithm for Large Language Modelscited by 1
  17. 2024Training Neural Networks for Modularity aids Interpretabilitycited by 1
  18. 2025Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learningcited by 1
  19. 2026Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshapingcited by 1
  20. 2024A mechanistically interpretable neural network for regulatory genomicscited by 0
  21. 2025Pruning Increases Orderedness in Recurrent Computationcited by 0
  22. 2025Interpretable Machine Learning for Life Expectancy Prediction: A Comparative Study of Linear Regression, Decision Tree, and Random Forestcited by 0
  23. 2026Robustness in sparse artificial neural networks trained with adaptive topologycited by 0
  24. 2026Engineering Verifiable Modularity in Transformers via Per-Layer Supervisioncited by 0
  25. 2026Enhancing AI Interpretability with Localised Architecturescited by 0
  26. 2026Circuit Condensation: Post-Training that Concentrates a Behavior's Causal Circuitcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Evaluating Brain-Inspired Modular Training in Automated Circuit Discovery for Mechanistic Interpretability
  2. 2026Mechanistic Interpretability for Neural Networks: Circuits, Sparse Features and Symbolic Reasoning
  3. 2024The Cognitive Revolution in Interpretability: From Explaining Behavior to Interpreting Representations and Algorithms
  4. 2024From Neurons to Neutrons: A Case Study in Interpretability
  5. 2026Mechanistically Interpretable Neural Encoding Reveals Fine-Grained Functional Selectivity in Human Visual Cortex
  6. 2025Mechanistic Interpretability Needs Philosophy
  7. 2025Interpretability in Parameter Space: Minimizing Mechanistic Description Length with Attribution-based Parameter Decomposition
  8. 2026Representation as a Bottleneck for Mechanistic Interpretability: The Manifestation Unit Protocol