# Explaining Convolutional Neural Networks through Attribution-Based Input Sampling and Block-Wise Feature Aggregation

arXiv: https://arxiv.org/abs/2010.00672 · HTML: https://mlsanity.com/paper/2010.00672/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Oct 2020.

## Built on by (most cited first, 13 of 13)

- [ViT-CX: Causal Explanation of Vision Transformers](https://mlsanity.com/paper/2211.03064/) (2022)
- [Learn-Explain-Reinforce: Counterfactual Reasoning and Its Guidance to Reinforce an Alzheimer's Disease Diagnosis Model](https://mlsanity.com/paper/2108.09451/) (2021)
- [Ada-SISE: Adaptive Semantic Input Sampling for Efficient Explanation of Convolutional Neural Networks](https://mlsanity.com/paper/2102.07799/) (2021)
- [Towards Trust of Explainable AI in Thyroid Nodule Diagnosis](https://mlsanity.com/paper/2303.04731/) (2023)
- [Randomly Removing 50% of Dimensions in Text Embeddings has Minimal Impact on Retrieval and Classification Tasks](https://mlsanity.com/paper/2508.17744/) (2025)
- [Learning Visual Explanations for DCNN-Based Image Classifiers Using an Attention Mechanism](https://mlsanity.com/paper/2209.11189/) (2022)
- [TAME: Attention Mechanism Based Feature Fusion for Generating Explanation Maps of Convolutional Neural Networks](https://mlsanity.com/paper/2301.07407/) (2023)
- [T-TAME: Trainable Attention Mechanism for Explaining Convolutional Networks and Vision Transformers](https://mlsanity.com/paper/2403.04523/) (2024)
- [A Novel Explainable Artificial Intelligence Model in Image Classification problem](https://mlsanity.com/paper/2307.04137/) (2023)
- [On The Coherence of Quantitative Evaluation of Visual Explanations](https://mlsanity.com/paper/2302.10764/) (2023)
- [EDDA: Explanation-driven Data Augmentation to Improve Explanation Faithfulness](https://mlsanity.com/paper/2105.14162/) (2021)
- [Rethinking gradient weights' influence over saliency map estimation](https://mlsanity.com/paper/2207.05374/) (2022)
- [P-TAME: Explain Any Image Classifier with Trained Perturbations](https://mlsanity.com/paper/2501.17813/) (2025)

## Abstract (arXiv)

As an emerging field in Machine Learning, Explainable AI (XAI) has been offering remarkable performance in interpreting the decisions made by Convolutional Neural Networks (CNNs). To achieve visual explanations for CNNs, methods based on class activation mapping and randomized input sampling have gained great popularity. However, the attribution methods based on these techniques provide lower resolution and blurry explanation maps that limit their explanation power. To circumvent this issue, visualization based on various layers is sought. In this work, we collect visualization maps from multiple layers of the model based on an attribution-based input sampling technique and aggregate them to reach a fine-grained and complete explanation. We also propose a layer selection strategy that applies to the whole family of CNN-based models, based on which our extraction framework is applied to visualize the last layers of each convolutional block of the model. Moreover, we perform an empirical analysis of the efficacy of derived lower-level information to enhance the represented attributions. Comprehensive experiments conducted on shallow and deep models trained on natural and industrial datasets, using both ground-truth and model-truth based evaluation metrics validate our proposed algorithm by meeting or outperforming the state-of-the-art methods in terms of explanation ability and visual quality, demonstrating that our method shows stability regardless of the size of objects or instances to be explained.