# DeepROCK: Error-controlled interaction detection in deep neural networks

arXiv: https://arxiv.org/abs/2309.15319 · HTML: https://mlsanity.com/paper/2309.15319/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 26 Sep 2023.

## Builds on (most cited first, 23 of 23)

- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Interpretation of Neural Networks is Fragile](https://mlsanity.com/paper/1710.10547/) (2017)
- [The (Un)reliability of saliency methods](https://mlsanity.com/paper/1711.00867/) (2017)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Explaining Explanations: Axiomatic Feature Interactions for Deep Networks](https://mlsanity.com/paper/2002.04138/) (2020)
- [Controlling the false discovery rate via knockoffs](https://mlsanity.com/paper/1404.5609/) (2014)
- [NODE-GAM: Neural Generalized Additive Model for Interpretable Deep Learning](https://mlsanity.com/paper/2106.01613/) (2021)
- [GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions](https://mlsanity.com/paper/2003.07132/) (2020)
- [Detecting Statistical Interactions from Neural Network Weights](https://mlsanity.com/paper/1705.04977/) (2017)
- [How does this interaction affect me? Interpretable attribution for feature interactions](https://mlsanity.com/paper/2006.10965/) (2020)
- [Iterative Random Forests to detect predictive and stable high-order interactions](https://mlsanity.com/paper/1706.08457/) (2017)
- [DeepPINK: reproducible feature selection in deep neural networks](https://mlsanity.com/paper/1809.01185/) (2018)
- [Deep Knockoffs](https://mlsanity.com/paper/1811.06687/) (2018)
- [Deep Direct Likelihood Knockoffs](https://mlsanity.com/paper/2007.15835/) (2020)
- [DANCE: Enhancing saliency maps using decoys](https://mlsanity.com/paper/2002.00526/) (2020)
- [Auto-Encoding Knockoff Generator for FDR Controlled Variable Selection](https://mlsanity.com/paper/1809.10765/) (2018)
- [Explaining Local, Global, And Higher-Order Interactions In Deep Learning](https://mlsanity.com/paper/2006.08601/) (2020)
- [Learning Global Pairwise Interactions with Bayesian Neural Networks](https://mlsanity.com/paper/1901.08361/) (2019)
- [Towards Interaction Detection Using Topological Analysis on Neural Networks](https://mlsanity.com/paper/2010.13015/) (2020)
- [Interpretable Artificial Intelligence through the Lens of Feature Interaction](https://mlsanity.com/paper/2103.03103/) (2021)

## Built on by (most cited first, 1 of 1)

- [Knockoffs-based False Discovery Rate Control and Simplification for Deep Neural Networks](https://mlsanity.com/paper/2606.04404/) (2026)

## Abstract (arXiv)

The complexity of deep neural networks (DNNs) makes them powerful but also makes them challenging to interpret, hindering their applicability in error-intolerant domains. Existing methods attempt to reason about the internal mechanism of DNNs by identifying feature interactions that influence prediction outcomes. However, such methods typically lack a systematic strategy to prioritize interactions while controlling confidence levels, making them difficult to apply in practice for scientific discovery and hypothesis validation. In this paper, we introduce a method, called DeepROCK, to address this limitation by using knockoffs, which are dummy variables that are designed to mimic the dependence structure of a given set of features while being conditionally independent of the response. Together with a novel DNN architecture involving a pairwise-coupling layer, DeepROCK jointly controls the false discovery rate (FDR) and maximizes statistical power. In addition, we identify a challenge in correctly controlling FDR using off-the-shelf feature interaction importance measures. DeepROCK overcomes this challenge by proposing a calibration procedure applied to existing interaction importance measures to make the FDR under control at a target level. Finally, we validate the effectiveness of DeepROCK through extensive experiments on simulated and real datasets.