# Revisiting the Importance of Individual Units in CNNs via Ablation

arXiv: https://arxiv.org/abs/1806.02891 · HTML: https://mlsanity.com/paper/1806.02891/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jun 2018.

## Built on by (most cited first, 30 of 32)

- [ReAct: Out-of-distribution Detection With Rectified Activations](https://mlsanity.com/paper/2111.12797/) (2021)
- [Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networks](https://mlsanity.com/paper/2207.13243/) (2022)
- [Differentiation and Specialization of Attention Heads via the Refined Local Learning Coefficient](https://mlsanity.com/paper/2410.02984/) (2024)
- [Neural Response Interpretation through the Lens of Critical Pathways](https://mlsanity.com/paper/2103.16886/) (2021)
- [Dimensions underlying the representational alignment of deep neural networks with humans](https://mlsanity.com/paper/2406.19087/) (2024)
- [Scientific Inference With Interpretable Machine Learning: Analyzing Models to Learn About Real-World Phenomena](https://mlsanity.com/paper/2206.05487/) (2022)
- [Decomposing and Editing Predictions by Modeling Model Computation](https://mlsanity.com/paper/2404.11534/) (2024)
- [Scale Alone Does not Improve Mechanistic Interpretability in Vision Models](https://mlsanity.com/paper/2307.05471/) (2023)
- [Network Pruning using Adaptive Exemplar Filters](https://mlsanity.com/paper/2101.07985/) (2021)
- [How and where does CLIP process negation?](https://mlsanity.com/paper/2407.10488/) (2024)
- [Quantifying Local Specialization in Deep Neural Networks](https://mlsanity.com/paper/2110.08058/) (2021)
- [Localizing Memorization in SSL Vision Encoders](https://mlsanity.com/paper/2409.19069/) (2024)
- [Deep Residual Networks Learn the Geodesic Curve in the Wasserstein Space](https://mlsanity.com/paper/2102.09235/) (2021)
- [Quantitative Performance Assessment of CNN Units via Topological Entropy Calculation](https://mlsanity.com/paper/2103.09716/) (2021)
- [On the special role of class-selective neurons in early training](https://mlsanity.com/paper/2305.17409/) (2023)
- [CFDP: Common Frequency Domain Pruning](https://mlsanity.com/paper/2306.04147/) (2023)
- [Automated Natural Language Explanation of Deep Visual Neurons with Large Models](https://mlsanity.com/paper/2310.10708/) (2023)
- [Update Selective Parameters: Federated Machine Unlearning Based on Model Explanation](https://mlsanity.com/paper/2406.12516/) (2024)
- [Granular Concept Circuits: Toward a Fine-Grained Circuit Discovery for Concept Representations](https://mlsanity.com/paper/2508.01728/) (2025)
- [Reduction of Class Activation Uncertainty with Background Information](https://mlsanity.com/paper/2305.03238/) (2023)
- [Artificial Neural Nets and the Representation of Human Concepts](https://mlsanity.com/paper/2312.05337/) (2023)
- [Estimating the Generalization in Deep Neural Networks via Sparsity](https://mlsanity.com/paper/2104.00851/) (2021)
- [DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation](https://mlsanity.com/paper/2104.02234/) (2021)
- [Neural Supervised Domain Adaptation by Augmenting Pre-trained Models with Random Units](https://mlsanity.com/paper/2106.04935/) (2021)
- [Probing Ranking LLMs: A Mechanistic Analysis for Information Retrieval](https://mlsanity.com/paper/2410.18527/) (2024)
- [A Survey on Interpretability in Visual Recognition](https://mlsanity.com/paper/2507.11099/) (2025)
- [Detecting and Learning Out-of-Distribution Data in the Open world: Algorithm and Theory](https://mlsanity.com/paper/2310.06221/) (2023)
- [Towards Efficient Target-Level Machine Unlearning Based on Essential Graph](https://mlsanity.com/paper/2406.10954/) (2024)
- [Canoe : A System for Collaborative Learning for Neural Nets](https://mlsanity.com/paper/2108.12124/) (2021)
- [Much Easier Said Than Done: Falsifying the Causal Relevance of Linear Decoding Methods](https://mlsanity.com/paper/2211.04367/) (2022)

## Abstract (arXiv)

We revisit the importance of the individual units in Convolutional Neural Networks (CNNs) for visual recognition. By conducting unit ablation experiments on CNNs trained on large scale image datasets, we demonstrate that, though ablating any individual unit does not hurt overall classification accuracy, it does lead to significant damage on the accuracy of specific classes. This result shows that an individual unit is specialized to encode information relevant to a subset of classes. We compute the correlation between the accuracy drop under unit ablation and various attributes of an individual unit such as class selectivity and weight L1 norm. We confirm that unit attributes such as class selectivity are a poor predictor for impact on overall accuracy as found previously in recent work citemorcos2018importance. However, our results show that class selectivity along with other attributes are good predictors of the importance of one unit to individual classes. We evaluate the impact of random rotation, batch normalization, and dropout to the importance of units to specific classes. Our results show that units with high selectivity play an important role in network classification power at the individual class level. Understanding and interpreting the behavior of these units is necessary and meaningful.