mlsanityData to 14 Oct 2026

Towards a fuller understanding of neurons with Clustered Compositional Explanations

arXiv 2310.18443 ↗First posted 27 Oct 2023cs.LG cs.AIcited by 15

Not on the map. Why?

Abstract (arXiv)

Compositional Explanations is a method for identifying logical formulas of concepts that approximate the neurons' behavior. However, these explanations are linked to the small spectrum of neuron activations (i.e., the highest ones) used to check the alignment, thus lacking completeness. In this paper, we propose a generalization, called Clustered Compositional Explanations, that combines Compositional Explanations with clustering and a novel search heuristic to approximate a broader spectrum of the neurons' behavior. We define and address the problems connected to the application of these methods to multiple ranges of activations, analyze the insights retrievable by using our algorithm, and propose desiderata qualities that can be used to study the explanations returned by different algorithms.

Key facts. Cited by 15 papers in the map (data through 14 Oct 2026). Builds on 23 of its 46 references that are in the map. First posted on arXiv 27 Oct 2023.

Builds on

23 of 46 references.

References · 23

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,069
  5. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 708
  6. 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 403
  8. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 255
  9. 2016Synthesizing the preferred inputs for neurons in neural networks via deep generator networkscited by 181
  10. 2014Object Detectors Emerge in Deep Scene CNNscited by 179
  11. 2020Compositional Explanations of Neuronscited by 172
  12. 2022CLIP-Dissect: Automatic Description of Neuron Representations in Deep Vision Networkscited by 147
11 more
  1. 2018What Is One Grain of Sand in the Desert? Analyzing Individual Neurons in Deep NLP Modelscited by 124
  2. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  3. 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
  4. 2021Neuron-level Interpretation of Deep NLP Models: A Surveycited by 81
  5. 2022Finding Skill Neurons in Pre-trained Transformer-based Language Modelscited by 66
  6. 2021On the Pitfalls of Analyzing Individual Neurons in Language Modelscited by 62
  7. 2020Intrinsic Probing through Dimension Selectioncited by 45
  8. 2019Discovery of Natural Language Concepts in Individual Units of CNNscited by 21
  9. 2019On the Units of GANs (Extended Abstract)cited by 19
  10. 2021Detection Accuracy for Evaluating Compositional Explanations of Unitscited by 3
  11. 2021Memory Wrap: a Data-Efficient and Interpretable Extension to Image Classification Modelscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 10; 2026: 2.

15 papers.

Sort:

By area:

  1. 2024Linear Explanations for Individual Neuronscited by 19
  2. 2025Disentangling Polysemantic Channels in Convolutional Neural Networkscited by 6
  3. 2024Explaining the Implicit Neural Canvas: Connecting Pixels to Neurons by Tracing their Contributionscited by 5
  4. 2025Neurons Speak in Ranges: Breaking Free from Discrete Neuronal Attributioncited by 5
  5. 2025LogicCBMs: Logic-Enhanced Concept-Based Learningcited by 5
  6. 2025Neuron-Guided Interpretation of Code LLMs: Where, Why, and How?cited by 5
  7. 2025Evaluating Neuron Explanations: A Unified Framework with Sanity Checkscited by 4
  8. 2025NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactionscited by 3
  9. 2025Faithful and Stable Neuron Explanations for Trustworthy Mechanistic Interpretabilitycited by 3
  10. 2024Towards Utilising a Range of Neural Activations for Comprehending Representational Associationscited by 2
  11. 2026What is Missing? Explaining Neurons Activated by Absent Conceptscited by 1
  12. 2025Beyond Top Activations: Efficient and Reliable Crowdsourced Evaluation of Automated Interpretabilitycited by 0
3 more
  1. 2025Open Vocabulary Compositional Explanations for Neuron Alignmentcited by 0
  2. 2025Guaranteed Optimal Compositional Explanations for Neuronscited by 0
  3. 2026Select, Hypothesize and Verify: Towards Verified Neuron Concept Interpretationcited by 0