# Towards Explanation of DNN-based Prediction with Guided Feature Inversion

arXiv: https://arxiv.org/abs/1804.00506 · HTML: https://mlsanity.com/paper/1804.00506/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2018.

## Built on by (most cited first, 30 of 31)

- [From Anecdotal Evidence to Quantitative Evaluation Methods: A Systematic Review on Evaluating Explainable AI](https://mlsanity.com/paper/2201.08164/) (2022)
- [On Explainability of Graph Neural Networks via Subgraph Explorations](https://mlsanity.com/paper/2102.05152/) (2021)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Towards Self-Explainable Graph Neural Network](https://mlsanity.com/paper/2108.12055/) (2021)
- [FunnyBirds: A Synthetic Vision Dataset for a Part-Based Analysis of Explainable AI Methods](https://mlsanity.com/paper/2308.06248/) (2023)
- [HYDRA: Hypergradient Data Relevance Analysis for Interpreting Deep Neural Networks](https://mlsanity.com/paper/2102.02515/) (2021)
- [FlowX: Towards Explainable Graph Neural Networks via Message Flows](https://mlsanity.com/paper/2206.12987/) (2022)
- [DeepAID: Interpreting and Improving Deep Learning-based Anomaly Detection in Security Applications](https://mlsanity.com/paper/2109.11495/) (2021)
- [Private Graph Extraction via Feature Explanations](https://mlsanity.com/paper/2206.14724/) (2022)
- [Towards Prototype-Based Self-Explainable Graph Neural Network](https://mlsanity.com/paper/2210.01974/) (2022)
- [Building Reliable Explanations of Unreliable Neural Networks: Locally Smoothing Perspective of Model Interpretation](https://mlsanity.com/paper/2103.14332/) (2021)
- [Sum-of-Parts: Self-Attributing Neural Networks with End-to-End Learning of Feature Groups](https://mlsanity.com/paper/2310.16316/) (2023)
- [Generalizable and Explainable Deep Learning for Medical Image Computing: An Overview](https://mlsanity.com/paper/2503.08420/) (2025)
- [Concept-Centric Token Interpretation for Vector-Quantized Generative Models](https://mlsanity.com/paper/2506.00698/) (2025)
- [A Comprehensive and Reliable Feature Attribution Method: Double-sided Remove and Reconstruct (DoRaR)](https://mlsanity.com/paper/2310.17945/) (2023)
- [AdaptGrad: Adaptive Sampling to Reduce Noise](https://mlsanity.com/paper/2410.07711/) (2024)
- [Attribution Explanations for Deep Neural Networks: A Theoretical Perspective](https://mlsanity.com/paper/2508.07636/) (2025)
- [An Embedded Feature Selection Framework for Control](https://mlsanity.com/paper/2206.11064/) (2022)
- [Lateral Movement Detection Using User Behavioral Analysis](https://mlsanity.com/paper/2208.13524/) (2022)
- [Ensemble Interpretation: A Unified Method for Interpretable Machine Learning](https://mlsanity.com/paper/2312.06255/) (2023)
- [Feature CAM: Interpretable AI in Image Classification](https://mlsanity.com/paper/2403.05658/) (2024)
- [ConcreTizer: Model Inversion Attack via Occupancy Classification and Dispersion Control for 3D Point Cloud Restoration](https://mlsanity.com/paper/2503.06986/) (2025)
- [Layer-based Composite Reputation Bootstrapping](https://mlsanity.com/paper/2102.09951/) (2021)
- [Understanding of Kernels in CNN Models by Suppressing Irrelevant Visual Features in Images](https://mlsanity.com/paper/2108.11054/) (2021)
- [Towards Efficient Visual Simplification of Computational Graphs in Deep Neural Networks](https://mlsanity.com/paper/2212.10774/) (2022)
- [On the Adversarial Inversion of Deep Biometric Representations](https://mlsanity.com/paper/2304.05561/) (2023)
- [Interpretable Modeling of Deep Reinforcement Learning Driven Scheduling](https://mlsanity.com/paper/2403.16293/) (2024)
- [Generating Counterfactual Explanations Using Cardinality Constraints](https://mlsanity.com/paper/2404.07502/) (2024)
- [Beyond 5G Network Failure Classification for Network Digital Twin Using Graph Neural Network](https://mlsanity.com/paper/2406.06595/) (2024)
- [Axiomatization of Gradient Smoothing in Neural Networks](https://mlsanity.com/paper/2407.00371/) (2024)

## Abstract (arXiv)

While deep neural networks (DNN) have become an effective computational tool, the prediction results are often criticized by the lack of interpretability, which is essential in many real-world applications such as health informatics. Existing attempts based on local interpretations aim to identify relevant features contributing the most to the prediction of DNN by monitoring the neighborhood of a given input. They usually simply ignore the intermediate layers of the DNN that might contain rich information for interpretation. To bridge the gap, in this paper, we propose to investigate a guided feature inversion framework for taking advantage of the deep architectures towards effective interpretation. The proposed framework not only determines the contribution of each feature in the input but also provides insights into the decision-making process of DNN models. By further interacting with the neuron of the target category at the output layer of the DNN, we enforce the interpretation result to be class-discriminative. We apply the proposed interpretation model to different CNN architectures to provide explanations for image data and conduct extensive experiments on ImageNet and PASCAL VOC07 datasets. The interpretation results demonstrate the effectiveness of our proposed framework in providing class-discriminative interpretation for DNN-based prediction.