# Analyzing Classifiers: Fisher Vectors and Deep Neural Networks

arXiv: https://arxiv.org/abs/1512.00172 · HTML: https://mlsanity.com/paper/1512.00172/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Dec 2015.

## Built on by (most cited first, 24 of 24)

- [What Do We Want From Explainable Artificial Intelligence (XAI)? -- A Stakeholder Perspective on XAI and a Conceptual Model Guiding Interdisciplinary XAI Research](https://mlsanity.com/paper/2102.07817/) (2021)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAy](https://mlsanity.com/paper/2106.13200/) (2021)
- [Learning Support and Trivial Prototypes for Interpretable Image Classification](https://mlsanity.com/paper/2301.04011/) (2023)
- [NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights](https://mlsanity.com/paper/2106.10185/) (2021)
- [Explaining Bayesian Neural Networks](https://mlsanity.com/paper/2108.10346/) (2021)
- [DORA: Exploring Outlier Representations in Deep Neural Networks](https://mlsanity.com/paper/2206.04530/) (2022)
- [Mixture of Gaussian-distributed Prototypes with Generative Modelling for Interpretable and Trustworthy Image Recognition](https://mlsanity.com/paper/2312.00092/) (2023)
- [How explainable are adversarially-robust CNNs?](https://mlsanity.com/paper/2205.13042/) (2022)
- [Visualizing and Understanding Contrastive Learning](https://mlsanity.com/paper/2206.09753/) (2022)
- [A New Perspective on Evaluation Methods for Explainable Artificial Intelligence (XAI)](https://mlsanity.com/paper/2307.14246/) (2023)
- [Interpretability of Machine Learning Methods Applied to Neuroimaging](https://mlsanity.com/paper/2204.07005/) (2022)
- [Are explainable AI (XAI) evaluation strategies aligned? Comparing subjective, objective, and mathematical evaluation measures using saliency maps](https://mlsanity.com/paper/2504.17023/) (2025)
- [Applications of interpretable deep learning in neuroimaging: a comprehensive review](https://mlsanity.com/paper/2406.17792/) (2024)
- [How Well Do Feature-Additive Explainers Explain Feature-Additive Predictors?](https://mlsanity.com/paper/2310.18496/) (2023)
- [Explainability-aided Domain Generalization for Image Classification](https://mlsanity.com/paper/2104.01742/) (2021)
- [Explainable multiple abnormality classification of chest CT volumes](https://mlsanity.com/paper/2111.12215/) (2021)
- [Illuminating Salient Contributions in Neuron Activation with Attribution Equilibrium](https://mlsanity.com/paper/2205.11109/) (2022)
- [On The Coherence of Quantitative Evaluation of Visual Explanations](https://mlsanity.com/paper/2302.10764/) (2023)
- [How Does a Deep Neural Network Look at Lexical Stress in English Words?](https://mlsanity.com/paper/2508.07229/) (2025)
- [[Reproducibility Report] Explainable Deep One-Class Classification](https://mlsanity.com/paper/2206.02598/) (2022)
- [EXPRESSNET: An Explainable Residual Slim Network for Fingerprint Presentation Attack Detection](https://mlsanity.com/paper/2305.09397/) (2023)
- [Hardness of Deceptive Certificate Selection](https://mlsanity.com/paper/2306.04505/) (2023)
- [A Vulnerability of Attribution Methods Using Pre-Softmax Scores](https://mlsanity.com/paper/2307.03305/) (2023)

## Abstract (arXiv)

Fisher Vector classifiers and Deep Neural Networks (DNNs) are popular and successful algorithms for solving image classification problems. However, both are generally considered `black box' predictors as the non-linear transformations involved have so far prevented transparent and interpretable reasoning. Recently, a principled technique, Layer-wise Relevance Propagation (LRP), has been developed in order to better comprehend the inherent structured reasoning of complex nonlinear classification models such as Bag of Feature models or DNNs. In this paper we (1) extend the LRP framework also for Fisher Vector classifiers and then use it as analysis tool to (2) quantify the importance of context for classification, (3) qualitatively compare DNNs against FV classifiers in terms of important image regions and (4) detect potential flaws and biases in data. All experiments are performed on the PASCAL VOC 2007 data set.