# PURE: Turning Polysemantic Neurons Into Pure Features by Identifying Relevant Circuits

arXiv: https://arxiv.org/abs/2404.06453 · HTML: https://mlsanity.com/paper/2404.06453/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability.
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 9 Apr 2024.

## Placed in

- Mechanistic interpretability / [Circuits and mechanistic analysis](https://mlsanity.com/mechinterp/circuits-and-mechanistic-analysis/)

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation](https://mlsanity.com/paper/2306.07304/) (2023)
- [Multifaceted Feature Visualization: Uncovering the Different Types of Features Learned By Each Neuron in Deep Neural Networks](https://mlsanity.com/paper/1602.03616/) (2016)
- [Multi-dimensional concept discovery (MCD): A unifying framework with completeness guarantees](https://mlsanity.com/paper/2301.11911/) (2023)
- [Identifying Interpretable Subspaces in Image Representations](https://mlsanity.com/paper/2307.10504/) (2023)
- [Labeling Neural Representations with Inverse Recognition](https://mlsanity.com/paper/2311.13594/) (2023)
- [Understanding the (Extra-)Ordinary: Validating Deep Model Decisions with Prototypical Concept-based Explanations](https://mlsanity.com/paper/2311.16681/) (2023)
- [DORA: Exploring Outlier Representations in Deep Neural Networks](https://mlsanity.com/paper/2206.04530/) (2022)

## Built on by (most cited first, 29 of 29)

- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [Mechanistic understanding and validation of large AI models with SemanticLens](https://mlsanity.com/paper/2501.05398/) (2025)
- [Style-Specific Neurons for Steering LLMs in Text Style Transfer](https://mlsanity.com/paper/2410.00593/) (2024)
- [When Truthful Representations Flip Under Deceptive Instructions?](https://mlsanity.com/paper/2507.22149/) (2025)
- [ConSim: Measuring Concept-Based Explanations' Effectiveness with Automated Simulatability](https://mlsanity.com/paper/2501.05855/) (2025)
- [CoE: Chain-of-Explanation via Automatic Visual Concept Circuit Description and Polysemanticity Quantification](https://mlsanity.com/paper/2503.15234/) (2025)
- [Disentangling Polysemantic Channels in Convolutional Neural Networks](https://mlsanity.com/paper/2504.12939/) (2025)
- [Certified Circuits: Stability Guarantees for Mechanistic Circuits](https://mlsanity.com/paper/2602.22968/) (2026)
- [Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence](https://mlsanity.com/paper/2505.11611/) (2025)
- [Spectral Superposition: A Theory of Feature Geometry](https://mlsanity.com/paper/2602.02224/) (2026)
- [Expand Neurons, Not Parameters](https://mlsanity.com/paper/2510.04500/) (2025)
- [Ensuring Medical AI Safety: Interpretability-Driven Detection and Mitigation of Spurious Model Behavior and Associated Data](https://mlsanity.com/paper/2501.13818/) (2025)
- [SAFR: Neuron Redistribution for Interpretability](https://mlsanity.com/paper/2501.16374/) (2025)
- [See What I Mean? CUE: A Cognitive Model of Understanding Explanations](https://mlsanity.com/paper/2506.14775/) (2025)
- [Mammo-SAE: Interpreting Breast Cancer Concept Learning with Sparse Autoencoders](https://mlsanity.com/paper/2507.15227/) (2025)
- [How Robust is Model Editing after Fine-Tuning? An Empirical Study on Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.18428/) (2025)
- [Disentangling Polysemantic Neurons with a Null-Calibrated Polysemanticity Index and Causal Patch Interventions](https://mlsanity.com/paper/2508.16950/) (2025)
- [FaCT: Faithful Concept Traces for Explaining Neural Network Decisions](https://mlsanity.com/paper/2510.25512/) (2025)
- [Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers](https://mlsanity.com/paper/2604.14477/) (2026)
- [Layerwise Convergence Fingerprints for Runtime Misbehavior Detection in Large Language Models](https://mlsanity.com/paper/2604.24542/) (2026)
- [Identifying Sub-networks in Neural Networks via Functionally Similar Representations](https://mlsanity.com/paper/2410.16484/) (2024)
- [Attack logics, not outputs: Towards efficient robustification of deep neural networks by falsifying concept-based properties](https://mlsanity.com/paper/2510.03320/) (2025)
- [Open Vocabulary Compositional Explanations for Neuron Alignment](https://mlsanity.com/paper/2511.20931/) (2025)
- [Guaranteed Optimal Compositional Explanations for Neurons](https://mlsanity.com/paper/2511.20934/) (2025)
- [Shortcut Mitigation via Spurious-Positive Samples](https://mlsanity.com/paper/2605.13340/) (2026)
- [Capability neq Interpretability: Human Interpretability of Vision Foundation Models](https://mlsanity.com/paper/2605.20337/) (2026)
- [Interpretability Without Tradeoffs: Disentangling Polysemanticity At Equal Predictive Performance](https://mlsanity.com/paper/2605.31304/) (2026)
- [Sparsity, Superposition, and Forgetting: A Mechanistic Study of Representation Retention in Continual Learning](https://mlsanity.com/paper/2606.20431/) (2026)
- [SPICE: Simple Polysemantic Feature Interpretation via Clustering-based Explanation](https://mlsanity.com/paper/2609.13198/) (2026)

## Abstract (arXiv)

The field of mechanistic interpretability aims to study the role of individual neurons in Deep Neural Networks. Single neurons, however, have the capability to act polysemantically and encode for multiple (unrelated) features, which renders their interpretation difficult. We present a method for disentangling polysemanticity of any Deep Neural Network by decomposing a polysemantic neuron into multiple monosemantic "virtual" neurons. This is achieved by identifying the relevant sub-graph ("circuit") for each "pure" feature. We demonstrate how our approach allows us to find and disentangle various polysemantic units of ResNet models trained on ImageNet. While evaluating feature visualizations using CLIP, our method effectively disentangles representations, improving upon methods based on neuron activations. Our code is available at https://github.com/maxdreyer/PURE.