mlsanityData to 7 Oct 2026

Clusterability in Neural Networks

arXiv 2103.03386 ↗First posted 4 Mar 2021cs.NEcited by 23

Not on the map. Why?

Abstract (arXiv)

The learned weights of a neural network have often been considered devoid of scrutable internal structure. In this paper, however, we look for structure in the form of clusterability: how well a network can be divided into groups of neurons with strong internal connectivity but weak external connectivity. We find that a trained neural network is typically more clusterable than randomly initialized networks, and often clusterable relative to random networks with the same distribution of weights. We also exhibit novel methods to promote clusterability in neural network training, and find that in multi-layer perceptrons they lead to more clusterable networks with little reduction in accuracy. Understanding and controlling the clusterability of neural networks will hopefully render their inner workings more interpretable to engineers by facilitating partitioning into meaningful clusters.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Mar 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 6; 2023: 4; 2024: 3; 2025: 3; 2026: 1.

23 papers.

Sort:

  1. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  2. 2021Visual Representation Learning Does Not Generalize Strongly Within the Same Domaincited by 60
  3. 2024Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficientcited by 39
  4. 2023Seeing is Believing: Brain-Inspired Modular Training for Mechanistic Interpretabilitycited by 38
  5. 2022Clustering units in neural networks: upstream vs downstream informationcited by 11
  6. 2021Quantifying Local Specialization in Deep Neural Networkscited by 10
  7. 2021Graph-Based Similarity of Neural Network Representationscited by 7
  8. 2021Dynamics of specialization in neural modules under resource constraintscited by 6
  9. 2021Modularity in Reinforcement Learning via Algorithmic Independence in Credit Assignmentcited by 6
  10. 2022Auditing Visualizations: Transparency Methods Struggle to Detect Anomalous Behaviorcited by 5
  11. 2022Pruning for Feature-Preserving Circuits in CNNscited by 4
  12. 2022Modular Federated Learningcited by 4
11 more
  1. 2023Modularity in Deep Learning: A Surveycited by 3
  2. 2023Understanding Sparse Neural Networks from their Topology via Multipartite Graph Representationscited by 2
  3. 2025Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.iicited by 2
  4. 2021Leveraging the Graph Structure of Neural Network Training Dynamicscited by 1
  5. 2024Geometric sparsification in recurrent neural networkscited by 1
  6. 2024Training Neural Networks for Modularity aids Interpretabilitycited by 1
  7. 2025Inducing, Detecting and Characterising Neural Modules: A Pipeline for Functional Interpretability in Reinforcement Learningcited by 1
  8. 2025Studying Cross-cluster Modularity in Neural Networkscited by 1
  9. 2022AI Model Utilization Measurements For Finding Class Encoding Patternscited by 0
  10. 2023Neural Sculpting: Uncovering hierarchically modular task structure in neural networks through pruning and network analysiscited by 0
  11. 2026RouterInterp: Understanding Superposed Specialisation in Mixture of Experts Routingcited by 0