mlsanityData to 7 Oct 2026

Disentangling Neuron Representations with Concept Vectors

arXiv 2304.09707 ↗First posted 19 Apr 2023cs.CV cs.LGcited by 28

Not on the map. Why?

Abstract (arXiv)

Mechanistic interpretability aims to understand how models store representations by breaking down neural networks into interpretable units. However, the occurrence of polysemantic neurons, or neurons that respond to multiple unrelated features, makes interpreting individual neurons challenging. This has led to the search for meaningful vectors, known as concept vectors, in activation space instead of individual neurons. The main contribution of this paper is a method to disentangle polysemantic neurons into concept vectors encapsulating distinct features. Our method can search for fine-grained concepts according to the user's desired level of concept separation. The analysis shows that polysemantic neurons can be disentangled into directions consisting of linear combinations of neurons. Our evaluations show that the concept vectors found encode coherent, human-understandable features.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 18 of its 30 references that are in the map. First posted on arXiv 19 Apr 2023.

Builds on

18 of 30 references.

References · 18

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  6. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,279
  9. 2022Mass-Editing Memory in a Transformercited by 976
  10. 2022Toy Models of Superpositioncited by 928
  11. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  12. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
6 more
  1. 2021Acquisition of Chess Knowledge in AlphaZerocited by 120
  2. 2016Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networkscited by 106
  3. 2020Rewriting a Deep Generative Modelcited by 105
  4. 2022Polysemanticity and Capacity in Neural Networkscited by 74
  5. 2022Interpreting Neural Networks through the Polytope Lenscited by 33
  6. 2022Engineering Monosemanticity in Toy Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 5; 2025: 11; 2026: 8.

28 papers.

Sort:

By area:

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2024Discover-then-Name: Task-Agnostic Concept Bottlenecks via Automated Concept Discoverycited by 91
  3. 2023Labeling Neural Representations with Inverse Recognitioncited by 31
  4. 2024Finetuning Foundation Models for Joint Analysis Optimizationcited by 18
  5. 2025CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantificationcited by 9
  6. 2024LLM-assisted Concept Discovery: Automatically Identifying and Explaining Neuron Functionscited by 6
  7. 2025Disentangling Polysemantic Channels in Convolutional Neural Networkscited by 6
  8. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  9. 2023Uncovering Unique Concept Vectors through Latent Space Decompositioncited by 5
  10. 2025NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactionscited by 3
  11. 2026Putting a Face to Forgetting: Continual Learning meets Mechanistic Interpretabilitycited by 3
  12. 2026Hierarchical Concept-based Interpretable Modelscited by 3
16 more
  1. 2023SPADE: Sparsity-Guided Debugging for Deep Neural Networkscited by 2
  2. 2023Generating Interpretable Networks using Hypernetworkscited by 2
  3. 2024Towards Utilising a Range of Neural Activations for Comprehending Representational Associationscited by 2
  4. 2025Mammo-SAE: Interpreting Breast Cancer Concept Learning with Sparse Autoencoderscited by 2
  5. 2025Discovering Intersectional Bias via Directional Alignment in Face Recognition Embeddingscited by 2
  6. 2025How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Modelscited by 1
  7. 2025FaCT: Faithful Concept Traces for Explaining Neural Network Decisionscited by 1
  8. 2026MonoLoss: A Training Objective for Interpretable Monosemantic Representationscited by 1
  9. 2026What is Missing? Explaining Neurons Activated by Absent Conceptscited by 1
  10. 2026From Local to Global to Mechanistic: An iERF-Centered Unified Framework for Interpreting Vision Modelscited by 1
  11. 2025Polysemy of Synthetic Neurons Towards a New Type of Explanatory Categorical Vector Spacescited by 0
  12. 2025Open Vocabulary Compositional Explanations for Neuron Alignmentcited by 0
  13. 2025Guaranteed Optimal Compositional Explanations for Neuronscited by 0
  14. 2026Correcting Influence: Unboxing LLM Outputs with Orthogonal Latent Spacescited by 0
  15. 2026Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performancecited by 0
  16. 2026Identifying Confusion Trends in Concept-based XAI for Multi-Label Classificationcited by 0