# Do semantic parts emerge in Convolutional Neural Networks?

arXiv: https://arxiv.org/abs/1607.03738 · HTML: https://mlsanity.com/paper/1607.03738/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jul 2016.

## Built on by (most cited first, 23 of 23)

- [Unsupervised Part Discovery from Contrastive Reconstruction](https://mlsanity.com/paper/2111.06349/) (2021)
- [Fed2: Feature-Aligned Federated Learning](https://mlsanity.com/paper/2111.14248/) (2021)
- [LOGICSEG: Parsing Visual Semantics with Neural Logic Learning and Reasoning](https://mlsanity.com/paper/2309.13556/) (2023)
- [Distillation of Diffusion Features for Semantic Correspondence](https://mlsanity.com/paper/2412.03512/) (2024)
- [3D Compositional Zero-shot Learning with DeCompositional Consensus](https://mlsanity.com/paper/2111.14673/) (2021)
- [Demystifying Unsupervised Semantic Correspondence Estimation](https://mlsanity.com/paper/2207.05054/) (2022)
- [HP-Capsule: Unsupervised Face Part Discovery by Hierarchical Parsing Capsule Network](https://mlsanity.com/paper/2203.10699/) (2022)
- [Part-guided Relational Transformers for Fine-grained Visual Recognition](https://mlsanity.com/paper/2212.13685/) (2022)
- [Evaluation of Saliency-based Explainability Method](https://mlsanity.com/paper/2106.12773/) (2021)
- [Learning Transferable Conceptual Prototypes for Interpretable Unsupervised Domain Adaptation](https://mlsanity.com/paper/2310.08071/) (2023)
- [Detection Accuracy for Evaluating Compositional Explanations of Units](https://mlsanity.com/paper/2109.07804/) (2021)
- [DeepEverest: Accelerating Declarative Top-K Queries for Deep Neural Network Interpretation](https://mlsanity.com/paper/2104.02234/) (2021)
- [Parsing Objects at a Finer Granularity: A Survey](https://mlsanity.com/paper/2212.13693/) (2022)
- [ADVISE: ADaptive Feature Relevance and VISual Explanations for Convolutional Neural Networks](https://mlsanity.com/paper/2203.01289/) (2022)
- [How Deep is Your Art: An Experimental Study on the Limits of Artistic Understanding in a Single-Task, Single-Modality Neural Network](https://mlsanity.com/paper/2203.16031/) (2022)
- [Interpreting Hidden Semantics in the Intermediate Layers of 3D Point Cloud Classification Neural Network](https://mlsanity.com/paper/2303.06652/) (2023)
- [LaFAM: Unsupervised Feature Attribution with Label-free Activation Maps](https://mlsanity.com/paper/2407.06059/) (2024)
- [Decomposing the Deep: Finding Class Specific Filters in Deep CNNs](https://mlsanity.com/paper/2112.07719/) (2021)
- [Discriminative Feature Learning through Feature Distance Loss](https://mlsanity.com/paper/2205.11606/) (2022)
- [Inching Towards Automated Understanding of the Meaning of Art: An Application to Computational Analysis of Mondrian's Artwork](https://mlsanity.com/paper/2302.00594/) (2022)
- [Towards the Characterization of Representations Learned via Capsule-based Network Architectures](https://mlsanity.com/paper/2305.05349/) (2023)
- [HOLMES: HOLonym-MEronym based Semantic inspection for Convolutional Image Classifiers](https://mlsanity.com/paper/2403.08536/) (2024)
- [SCAN: Visual Explanations with Self-Confidence and Analysis Networks](https://mlsanity.com/paper/2603.06523/) (2026)

## Abstract (arXiv)

Semantic object parts can be useful for several visual recognition tasks. Lately, these tasks have been addressed using Convolutional Neural Networks (CNN), achieving outstanding results. In this work we study whether CNNs learn semantic parts in their internal representation. We investigate the responses of convolutional filters and try to associate their stimuli with semantic parts. We perform two extensive quantitative analyses. First, we use ground-truth part bounding-boxes from the PASCAL-Part dataset to determine how many of those semantic parts emerge in the CNN. We explore this emergence for different layers, network depths, and supervision levels. Second, we collect human judgements in order to study what fraction of all filters systematically fire on any semantic part, even if not annotated in PASCAL-Part. Moreover, we explore several connections between discriminative power and semantics. We find out which are the most discriminative filters for object recognition, and analyze whether they respond to semantic parts or to other image patches. We also investigate the other direction: we determine which semantic parts are the most discriminative and whether they correspond to those parts emerging in the network. This enables to gain an even deeper understanding of the role of semantic parts in the network.