# Interpretable Artificial Intelligence through the Lens of Feature Interaction

arXiv: https://arxiv.org/abs/2103.03103 · HTML: https://mlsanity.com/paper/2103.03103/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 1 Mar 2021.

## Builds on (most cited first, 21 of 21)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)](https://mlsanity.com/paper/1711.11279/) (2017)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)
- [Detecting Statistical Interactions from Neural Network Weights](https://mlsanity.com/paper/1705.04977/) (2017)
- [How does this interaction affect me? Interpretable attribution for feature interactions](https://mlsanity.com/paper/2006.10965/) (2020)
- [Beyond Word Importance: Contextual Decomposition to Extract Interactions from LSTMs](https://mlsanity.com/paper/1801.05453/) (2018)
- [Hierarchical interpretations for neural network predictions](https://mlsanity.com/paper/1806.05337/) (2018)
- [Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection](https://mlsanity.com/paper/2006.10966/) (2020)
- [Towards Hierarchical Importance Attribution: Explaining Compositional Semantics for Neural Sequence Models](https://mlsanity.com/paper/1911.06194/) (2019)
- [Understanding Impacts of High-Order Loss Approximations and Features in Deep Learning Interpretation](https://mlsanity.com/paper/1902.00407/) (2019)
- [Can I trust you more? Model-Agnostic Hierarchical Explanations](https://mlsanity.com/paper/1812.04801/) (2018)

## Built on by (most cited first, 5 of 5)

- [Local MDI+: Local Feature Importances for Tree-Based Models](https://mlsanity.com/paper/2506.08928/) (2025)
- [FINCH: Locally Visualizing Higher-Order Feature Interactions in Black Box Models](https://mlsanity.com/paper/2503.16445/) (2025)
- [DeepROCK: Error-controlled interaction detection in deep neural networks](https://mlsanity.com/paper/2309.15319/) (2023)
- [Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark](https://mlsanity.com/paper/2207.14160/) (2022)
- [Error-controlled non-additive interaction discovery in machine learning models](https://mlsanity.com/paper/2408.17016/) (2024)

## Abstract (arXiv)

Interpretation of deep learning models is a very challenging problem because of their large number of parameters, complex connections between nodes, and unintelligible feature representations. Despite this, many view interpretability as a key solution to trustworthiness, fairness, and safety, especially as deep learning is applied to more critical decision tasks like credit approval, job screening, and recidivism prediction. There is an abundance of good research providing interpretability to deep learning models; however, many of the commonly used methods do not consider a phenomenon called "feature interaction." This work first explains the historical and modern importance of feature interactions and then surveys the modern interpretability methods which do explicitly consider feature interactions. This survey aims to bring to light the importance of feature interactions in the larger context of machine learning interpretability, especially in a modern context where deep learning models heavily rely on feature interactions.