# A Functional Information Perspective on Model Interpretation

arXiv: https://arxiv.org/abs/2206.05700 · HTML: https://mlsanity.com/paper/2206.05700/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 41 references that are in the map.
- First posted on arXiv 12 Jun 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [SmoothGrad: removing noise by adding noise](https://mlsanity.com/paper/1706.03825/) (2017)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Striving for Simplicity: The All Convolutional Net](https://mlsanity.com/paper/1412.6806/) (2014)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Recent Advances in Convolutional Neural Networks](https://mlsanity.com/paper/1512.07108/) (2015)
- [Not Just a Black Box: Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1605.01713/) (2016)
- [Learning to Explain: An Information-Theoretic Perspective on Model Interpretation](https://mlsanity.com/paper/1802.07814/) (2018)
- [A review of deep learning in medical imaging: Imaging traits, technology trends, case studies with progress highlights, and future promises](https://mlsanity.com/paper/2008.09104/) (2020)
- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [A Sensitivity Analysis of (and Practitioners' Guide to) Convolutional Neural Networks for Sentence Classification](https://mlsanity.com/paper/1510.03820/) (2015)
- [Very Deep Convolutional Neural Networks for Raw Waveforms](https://mlsanity.com/paper/1610.00087/) (2016)
- [A Theoretical Explanation for Perplexing Behaviors of Backpropagation-based Visualizations](https://mlsanity.com/paper/1805.07039/) (2018)
- [Removing Bias in Multi-modal Classifiers: Regularization by Maximizing Functional Entropies](https://mlsanity.com/paper/2010.10802/) (2020)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Visualization of Supervised and Self-Supervised Neural Networks via Attribution Guided Factorization](https://mlsanity.com/paper/2012.02166/) (2020)
- [Are VQA Systems RAD? Measuring Robustness to Augmented Data with Focused Interventions](https://mlsanity.com/paper/2106.04484/) (2021)
- [Latent Space Explanation by Intervention](https://mlsanity.com/paper/2112.04895/) (2021)
- [Learning a Metric Embedding for Face Recognition using the Multibatch Method](https://mlsanity.com/paper/1605.07270/) (2016)

## Built on by (most cited first, 5 of 5)

- [Faithful Explanations of Black-box NLP Models Using LLM-generated Counterfactuals](https://mlsanity.com/paper/2310.00603/) (2023)
- [On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs](https://mlsanity.com/paper/2407.19200/) (2024)
- [Layer Collaboration in the Forward-Forward Algorithm](https://mlsanity.com/paper/2305.12393/) (2023)
- [An Additive Instance-Wise Approach to Multi-class Model Interpretation](https://mlsanity.com/paper/2207.03113/) (2022)
- [BELLA: Black box model Explanations by Local Linear Approximations](https://mlsanity.com/paper/2305.11311/) (2023)

## Abstract (arXiv)

Contemporary predictive models are hard to interpret as their deep nets exploit numerous complex relations between input elements. This work suggests a theoretical framework for model interpretability by measuring the contribution of relevant features to the functional entropy of the network with respect to the input. We rely on the log-Sobolev inequality that bounds the functional entropy by the functional Fisher information with respect to the covariance of the data. This provides a principled way to measure the amount of information contribution of a subset of features to the decision function. Through extensive experiments, we show that our method surpasses existing interpretability sampling-based methods on various data signals such as image, text, and audio.