# Fidelity of Interpretability Methods and Perturbation Artifacts in Neural Networks

arXiv: https://arxiv.org/abs/2203.02928 · HTML: https://mlsanity.com/paper/2203.02928/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 6 Mar 2022.

## Builds on (most cited first, 15 of 15)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Interpretable Explanations of Black Boxes by Meaningful Perturbation](https://mlsanity.com/paper/1704.03296/) (2017)
- [A Benchmark for Interpretability Methods in Deep Neural Networks](https://mlsanity.com/paper/1806.10758/) (2018)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Evaluating the visualization of what a Deep Neural Network has learned](https://mlsanity.com/paper/1509.06321/) (2015)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps](https://mlsanity.com/paper/1902.04893/) (2019)
- [Resisting Out-of-Distribution Data Problem in Perturbation of XAI](https://mlsanity.com/paper/2107.14000/) (2021)

## Built on by (most cited first, 7 of 7)

- [Finding the right XAI method -- A Guide for the Evaluation and Ranking of Explainable AI Methods in Climate Science](https://mlsanity.com/paper/2303.00652/) (2023)
- [Class-Discriminative Attention Maps for Vision Transformers](https://mlsanity.com/paper/2312.02364/) (2023)
- [False Sense of Security in Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2405.03820/) (2024)
- [Explain and Monitor Deep Learning Models for Computer Vision using Obz AI](https://mlsanity.com/paper/2508.18188/) (2025)
- [Regularizing Attention Scores with Bootstrapping](https://mlsanity.com/paper/2604.01339/) (2026)
- [AIM: Adversarial Information Masking for Faithfulness Evaluation of Saliency Maps](https://mlsanity.com/paper/2605.16905/) (2026)
- [Explaining Time Series Forecasting with Horizon-Resolved Attribution](https://mlsanity.com/paper/2609.12639/) (2026)

## Abstract (arXiv)

Despite excellent performance of deep neural networks (DNNs) in image classification, detection, and prediction, characterizing how DNNs make a given decision remains an open problem, resulting in a number of interpretability methods. Post-hoc interpretability methods primarily aim to quantify the importance of input features with respect to the class probabilities. However, due to the lack of ground truth and the existence of interpretability methods with diverse operating characteristics, evaluating these methods is a crucial challenge. A popular approach to evaluate interpretability methods is to perturb input features deemed important for a given prediction and observe the decrease in accuracy. However, perturbation itself may introduce artifacts. We propose a method for estimating the impact of such artifacts on the fidelity estimation by utilizing model accuracy curves from perturbing input features according to the Most Import First (MIF) and Least Import First (LIF) orders. Using the ResNet-50 trained on the ImageNet, we demonstrate the proposed fidelity estimation of four popular post-hoc interpretability methods.