# Comprehensible Convolutional Neural Networks via Guided Concept Learning

arXiv: https://arxiv.org/abs/2101.03919 · HTML: https://mlsanity.com/paper/2101.03919/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 40 references that are in the map.
- First posted on arXiv 11 Jan 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Show and Tell: A Neural Image Caption Generator](https://mlsanity.com/paper/1411.4555/) (2014)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Zero-Shot Learning -- A Comprehensive Evaluation of the Good, the Bad and the Ugly](https://mlsanity.com/paper/1707.00600/) (2017)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Detect What You Can: Detecting and Representing Objects using Holistic Models and Body Parts](https://mlsanity.com/paper/1406.2031/) (2014)
- [Learning Deep Representations of Fine-grained Visual Descriptions](https://mlsanity.com/paper/1605.05395/) (2016)
- [Multimodal Explanations: Justifying Decisions and Pointing to the Evidence](https://mlsanity.com/paper/1802.08129/) (2018)
- [Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/1710.00935/) (2017)
- [Generating Visual Explanations](https://mlsanity.com/paper/1603.08507/) (2016)
- [Looking for the Devil in the Details: Learning Trilinear Attention Sampling Network for Fine-grained Image Recognition](https://mlsanity.com/paper/1903.06150/) (2019)
- [Grounding Visual Explanations](https://mlsanity.com/paper/1807.09685/) (2018)
- [Interpreting CNNs via Decision Trees](https://mlsanity.com/paper/1802.00121/) (2018)
- [Interpretable and Accurate Fine-grained Recognition via Region Grouping](https://mlsanity.com/paper/2005.10411/) (2020)
- [Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answering](https://mlsanity.com/paper/1511.05234/) (2015)
- [Fine-Grained Representation Learning and Recognition by Exploiting Hierarchical Semantic Embedding](https://mlsanity.com/paper/1808.04505/) (2018)
- [Explaining Neural Networks Semantically and Quantitatively](https://mlsanity.com/paper/1812.07169/) (2018)
- [An unexpected unity among methods for interpreting model predictions](https://mlsanity.com/paper/1611.07478/) (2016)

## Built on by (most cited first, 11 of 11)

- [Explainable Deep Learning Methods in Medical Image Classification: A Survey](https://mlsanity.com/paper/2205.04766/) (2022)
- [Coherent Concept-based Explanations in Medical Image and Its Application to Skin Lesion Diagnosis](https://mlsanity.com/paper/2304.04579/) (2023)
- [Faithful and Plausible Natural Language Explanations for Image Classification: A Pipeline Approach](https://mlsanity.com/paper/2407.20899/) (2024)
- [Towards Learning and Explaining Indirect Causal Effects in Neural Networks](https://mlsanity.com/paper/2303.13850/) (2023)
- [Seeing in Words: Learning to Classify through Language Bottlenecks](https://mlsanity.com/paper/2307.00028/) (2023)
- [Concept-based Explainable Malignancy Scoring on Pulmonary Nodules in CT Images](https://mlsanity.com/paper/2405.17483/) (2024)
- [SEZ-HARN: Self-Explainable Zero-shot Human Activity Recognition Network](https://mlsanity.com/paper/2507.00050/) (2025)
- [Towards Fully Interpretable Deep Neural Networks: Are We There Yet?](https://mlsanity.com/paper/2106.13164/) (2021)
- [ParseCaps: An Interpretable Parsing Capsule Network for Medical Image Diagnosis](https://mlsanity.com/paper/2411.01564/) (2024)
- [Decomposing the Deep: Finding Class Specific Filters in Deep CNNs](https://mlsanity.com/paper/2112.07719/) (2021)
- [Uncertainty-Aware Information Pursuit for Interpretable and Reliable Medical Image Analysis](https://mlsanity.com/paper/2506.16742/) (2025)

## Abstract (arXiv)

Learning concepts that are consistent with human perception is important for Deep Neural Networks to win end-user trust. Post-hoc interpretation methods lack transparency in the feature representations learned by the models. This work proposes a guided learning approach with an additional concept layer in a CNN- based architecture to learn the associations between visual features and word phrases. We design an objective function that optimizes both prediction accuracy and semantics of the learned feature representations. Experiment results demonstrate that the proposed model can learn concepts that are consistent with human perception and their corresponding contributions to the model decision without compromising accuracy. Further, these learned concepts are transferable to new classes of objects that have similar concepts.