# Interpreting Undesirable Pixels for Image Classification on Black-Box Models

arXiv: https://arxiv.org/abs/1909.12446 · HTML: https://mlsanity.com/paper/1909.12446/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Sep 2019.

## Built on by (most cited first, 2 of 2)

- [Resisting Out-of-Distribution Data Problem in Perturbation of XAI](https://mlsanity.com/paper/2107.14000/) (2021)
- [On the Effectiveness of Methods and Metrics for Explainable AI in Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2507.05916/) (2025)

## Abstract (arXiv)

In an effort to interpret black-box models, researches for developing explanation methods have proceeded in recent years. Most studies have tried to identify input pixels that are crucial to the prediction of a classifier. While this approach is meaningful to analyse the characteristic of blackbox models, it is also important to investigate pixels that interfere with the prediction. To tackle this issue, in this paper, we propose an explanation method that visualizes undesirable regions to classify an image as a target class. To be specific, we divide the concept of undesirable regions into two terms: (1) factors for a target class, which hinder that black-box models identify intrinsic characteristics of a target class and (2) factors for non-target classes that are important regions for an image to be classified as other classes. We visualize such undesirable regions on heatmaps to qualitatively validate the proposed method. Furthermore, we present an evaluation metric to provide quantitative results on ImageNet.