# Error-controlled non-additive interaction discovery in machine learning models

arXiv: https://arxiv.org/abs/2408.17016 · HTML: https://mlsanity.com/paper/2408.17016/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 71 references that are in the map.
- First posted on arXiv 30 Aug 2024.

## Builds on (most cited first, 25 of 25)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [The Mythos of Model Interpretability](https://mlsanity.com/paper/1606.03490/) (2016)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning](https://mlsanity.com/paper/2106.01613/) (2021)
- [Detecting Statistical Interactions from Neural Network Weights](https://mlsanity.com/paper/1705.04977/) (2017)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [DeepPINK: reproducible feature selection in deep neural networks](https://mlsanity.com/paper/1809.01185/) (2018)
- [Deep Knockoffs](https://mlsanity.com/paper/1811.06687/) (2018)
- [DANCE: Enhancing saliency maps using decoys](https://mlsanity.com/paper/2002.00526/) (2020)
- [Auto-Encoding Knockoff Generator for FDR Controlled Variable Selection](https://mlsanity.com/paper/1809.10765/) (2018)
- [Explaining Local, Global, And Higher-Order Interactions In Deep Learning](https://mlsanity.com/paper/2006.08601/) (2020)
- [Learning Global Pairwise Interactions with Bayesian Neural Networks](https://mlsanity.com/paper/1901.08361/) (2019)
- [Interpretable Artificial Intelligence through the Lens of Feature Interaction](https://mlsanity.com/paper/2103.03103/) (2021)
- [When Knockoffs fail: diagnosing and fixing non-exchangeability of Knockoffs](https://mlsanity.com/paper/2407.06892/) (2024)

## Abstract (arXiv)

Machine learning (ML) models are powerful tools for detecting complex patterns within data, yet their "black box" nature limits their interpretability, hindering their use in critical domains like healthcare and finance. To address this challenge, interpretable ML methods have been developed to explain how features influence model predictions. However, these methods often focus on univariate feature importance, overlooking the complex interactions between features that ML models are capable of capturing. Recognizing this limitation, recent efforts have aimed to extend these methods to discover feature interactions, but existing approaches struggle with robustness and error control, especially under data perturbations. In this study, we introduce Diamond, a novel method for trustworthy feature interaction discovery. Diamond uniquely integrates the model-X knockoffs framework to control the false discovery rate (FDR), ensuring that the proportion of falsely discovered interactions remains low. A key innovation in Diamond is its non-additivity distillation procedure, which refines existing interaction importance measures to distill non-additive interaction effects, ensuring that FDR control is maintained. This approach addresses the limitations of off-the-shelf interaction measures, which, when used naively, can lead to inaccurate discoveries. Diamond's applicability spans a wide range of ML models, including deep neural networks, transformer models, tree-based models, and factorization-based models. Our empirical evaluations on both simulated and real datasets across various biomedical studies demonstrate Diamond's utility in enabling more reliable data-driven scientific discoveries. This method represents a significant step forward in the deployment of ML models for scientific innovation and hypothesis generation.