mlsanityData to 7 Oct 2026

Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks

arXiv 2010.02066 ↗First posted 5 Oct 2020cs.NE cs.AI cs.LGcited by 97

Not on the map. Why?

Abstract (arXiv)

Neural networks (NNs) whose subnetworks implement reusable functions are expected to offer numerous advantages, including compositionality through efficient recombination of functional building blocks, interpretability, preventing catastrophic interference, etc. Understanding if and how NNs are modular could provide insights into how to improve them. Current inspection methods, however, fail to link modules to their functionality. In this paper, we present a novel method based on learning binary weight masks to identify individual weights and subnets responsible for specific functions. Using this powerful tool, we contribute an extensive study of emerging modularity in NNs that covers several standard architectures and datasets. We demonstrate how common NNs fail to reuse submodules and offer new insights into the related issue of systematic generalization on language tasks.

Key facts. Cited by 97 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 11; 2022: 10; 2023: 25; 2024: 23; 2025: 16; 2026: 12.

97 papers.

Sort:

By area:

  1. 2021Editing Factual Knowledge in Language Modelscited by 548
  2. 2022Memory-Based Model Editing at Scalecited by 443
  3. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 237
  4. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  5. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 194
  6. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 128
  7. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  8. 2021The Devil is in the Detail: Simple Tricks Improve Systematic Generalization of Transformerscited by 117
  9. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  10. 2023Soft Merging of Experts with Adaptive Routingcited by 88
  11. 2024A Primer on the Inner Workings of Transformer-based Language Modelscited by 83
  12. 2023Modular Deep Learningcited by 82
38 more
  1. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  2. 2021Can Subnetwork Structure be the Key to Out-of-Distribution Generalization?cited by 75
  3. 2021Continual Learning via Local Module Compositioncited by 61
  4. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  5. 2022Advancing Model Pruning via Bi-level Optimizationcited by 55
  6. 2025MIB: A Mechanistic Interpretability Benchmarkcited by 49
  7. 2025Do Language Models Use Their Depth Efficiently?cited by 48
  8. 2024Recurrent Neural Networks Learn to Store and Generate Sequences using Non-Linear Representationscited by 45
  9. 2023Break It Down: Evidence for Structural Compositionality in Neural Networkscited by 44
  10. 2023Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretabilitycited by 38
  11. 2023Emergent Modularity in Pre-trained Transformerscited by 37
  12. 2022Is a Modular Architecture Enough?cited by 35
  13. 2021Sparse Interventions in Language Models with Differentiable Maskingcited by 34
  14. 2024Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Smallcited by 34
  15. 2022How to Reuse and Compose Knowledge for a Lifetime of Tasks: A Survey on Continual Learning and Functional Compositioncited by 27
  16. 2024MrT5: Dynamic Token Merging for Efficient Byte-level Language Modelscited by 27
  17. 2024On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMscited by 24
  18. 2025How Do Transformers Learn Variable Binding in Symbolic Programs?cited by 19
  19. 2023Unlocking Emergent Modularity in Large Language Modelscited by 17
  20. 2022Training Debiased Subnetworks with Contrastive Weight Pruningcited by 16
  21. 2023Discovering Knowledge-Critical Subnetworks in Pretrained Language Modelscited by 16
  22. 2024How and where does CLIP process negation?cited by 16
  23. 2024A Reply to Makelov et al. (2023)'s "Interpretability Illusion" Argumentscited by 15
  24. 2025Stochastic Parameter Decompositioncited by 14
  25. 2023Examining Modularity in Multilingual LMs via Language-Specialized Subnetworkscited by 13
  26. 2023Winning Prize Comes from Losing Tickets: Improve Invariant Learning by Exploring Variant Parameters for Out-of-Distribution Generalizationcited by 12
  27. 2023Uncovering Intermediate Variables in Transformers using Circuit Probingcited by 12
  28. 2022Clustering units in neural networks: upstream vs downstream informationcited by 11
  29. 2024Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularitycited by 11
  30. 2025Are formal and functional linguistic mechanisms dissociated in language models?cited by 11
  31. 2026From Directions to Regions: Decomposing Activations in Language Models via Local Geometrycited by 10
  32. 2021Neural Network Module Decomposition and Recompositioncited by 9
  33. 2023NeuroSurgeon: A Toolkit for Subnetwork Analysiscited by 9
  34. 2024Out of Spuriousity: Improving Robustness to Spurious Correlations without Group Annotationscited by 9
  35. 2024Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Modelscited by 9
  36. 2023Causal interventions expose implicit situation models for commonsense language understandingcited by 7
  37. 2023Regularized Mask Tuning: Uncovering Hidden Knowledge in Pre-trained Vision-Language Modelscited by 7
  38. 2025LLMs Process Lists With General Filter Headscited by 7

50 most cited of 97 All 97 (JSON)