# Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNs

arXiv: https://arxiv.org/abs/2304.13312 · HTML: https://mlsanity.com/paper/2304.13312/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 26 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps](https://mlsanity.com/paper/1312.6034/) (2013)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Understanding Neural Networks Through Deep Visualization](https://mlsanity.com/paper/1506.06579/) (2015)
- [Real Time Image Saliency for Black Box Classifiers](https://mlsanity.com/paper/1705.07857/) (2017)
- [The Shapley Taylor Interaction Index](https://mlsanity.com/paper/1902.05622/) (2019)
- [Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximation](https://mlsanity.com/paper/1903.10992/) (2019)
- [A Unified Approach to Interpreting and Boosting Adversarial Transferability](https://mlsanity.com/paper/2010.04055/) (2020)
- [Faith-Shap: The Faithful Shapley Interaction Index](https://mlsanity.com/paper/2203.00870/) (2022)
- [Discovering and Explaining the Representation Bottleneck of DNNs](https://mlsanity.com/paper/2111.06236/) (2021)
- [Defining and Quantifying the Emergence of Sparse Concepts in DNNs](https://mlsanity.com/paper/2111.06206/) (2021)
- [Interpreting and Boosting Dropout from a Game-Theoretic View](https://mlsanity.com/paper/2009.11729/) (2020)
- [Does a Neural Network Really Encode Symbolic Concepts?](https://mlsanity.com/paper/2302.13080/) (2023)
- [Explaining Generalization Power of a DNN Using Interactive Concepts](https://mlsanity.com/paper/2302.13091/) (2023)
- [HarsanyiNet: Computing Accurate Shapley Values in a Single Forward Propagation](https://mlsanity.com/paper/2304.01811/) (2023)
- [A Game-Theoretic Taxonomy of Visual Concepts in DNNs](https://mlsanity.com/paper/2106.10938/) (2021)
- [A Unified Taylor Framework for Revisiting Attribution Methods](https://mlsanity.com/paper/2008.09695/) (2020)
- [Interpreting Attributions and Interactions of Adversarial Attacks](https://mlsanity.com/paper/2108.06895/) (2021)
- [Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferability](https://mlsanity.com/paper/2207.11694/) (2022)
- [Understanding and Unifying Fourteen Attribution Methods with Taylor Interactions](https://mlsanity.com/paper/2303.01506/) (2023)
- [Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?](https://mlsanity.com/paper/2304.01083/) (2023)
- [A Hypothesis for the Aesthetic Appreciation in Neural Networks](https://mlsanity.com/paper/2108.02646/) (2021)

## Built on by (most cited first, 12 of 12)

- [ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMs](https://mlsanity.com/paper/2505.17495/) (2025)
- [Defining and Extracting generalizable interaction primitives from DNNs](https://mlsanity.com/paper/2401.16318/) (2024)
- [Towards the Dynamics of a DNN Learning Symbolic Interactions](https://mlsanity.com/paper/2407.19198/) (2024)
- [Where We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Models](https://mlsanity.com/paper/2305.01939/) (2023)
- [Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Features](https://mlsanity.com/paper/2405.10262/) (2024)
- [Layerwise Change of Knowledge in Neural Networks](https://mlsanity.com/paper/2409.08712/) (2024)
- [Quantifying In-Context Reasoning Effects and Memorization Effects in LLMs](https://mlsanity.com/paper/2405.11880/) (2024)
- [Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions](https://mlsanity.com/paper/2502.10162/) (2025)
- [Towards Attributions of Input Variables in a Coalition](https://mlsanity.com/paper/2309.13411/) (2023)
- [Fairness of Explanations in Artificial Intelligence (AI): A Unifying Framework, Axioms, and Future Direction toward Responsible AI](https://mlsanity.com/paper/2605.09852/) (2026)
- [Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspective](https://mlsanity.com/paper/2605.17967/) (2026)
- [How Does Parameter Pruning Reshape DNN Representations? An Interaction-Driven Exploration](https://mlsanity.com/paper/2609.06483/) (2026)

## Abstract (arXiv)

In this technical note, we aim to explain a deep neural network (DNN) by quantifying the encoded interactions between input variables, which reflects the DNN's inference logic. Specifically, we first rethink the definition of interactions, and then formally define faithfulness and conciseness for interaction-based explanation. To this end, we propose two kinds of interactions, i.e., the AND interaction and the OR interaction. For faithfulness, we prove the uniqueness of the AND (OR) interaction in quantifying the effect of the AND (OR) relationship between input variables. Besides, based on AND-OR interactions, we design techniques to boost the conciseness of the explanation, while not hurting the faithfulness. In this way, the inference logic of a DNN can be faithfully and concisely explained by a set of symbolic concepts.