Revisiting Generalization Power of a DNN in Terms of Symbolic Interactions
Not on the map. Why?
Abstract (arXiv)
This paper aims to analyze the generalization power of deep neural networks (DNNs) from the perspective of interactions. Unlike previous analysis of a DNN's generalization power in a highdimensional feature space, we find that the generalization power of a DNN can be explained as the generalization power of the interactions. We found that the generalizable interactions follow a decay-shaped distribution, while non-generalizable interactions follow a spindle-shaped distribution. Furthermore, our theory can effectively disentangle these two types of interactions from a DNN. We have verified that our theory can well match real interactions in a DNN in experiments.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 28 of its 36 references that are in the map. First posted on arXiv 14 Feb 2025.
Builds on
28 of 36 references.
References · 28
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
- 2014Explaining and Harnessing Adversarial Examplescited by 7,364
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
- 2017Exploring Generalization in Deep Learningcited by 600
- 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
- 2017Real Time Image Saliency for Black Box Classifierscited by 246
- 2019Sharper bounds for uniformly stable algorithmscited by 93
- 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
16 more
- 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
- 2020Toward Better Generalization Bounds with Locally Elastic Stabilitycited by 42
- 2021Discovering and Explaining the Representation Bottleneck of DNNscited by 42
- 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
- 2021Towards a Unified Information-Theoretic Framework for Generalizationcited by 30
- 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
- 2023Explaining Generalization Power of a DNN Using Interactive Conceptscited by 25
- 2022Learning sparse features can lead to overfitting in neural networkscited by 23
- 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
- 2024Towards the Dynamics of a DNN Learning Symbolic Interactionscited by 13
- 2023Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNscited by 12
- 2023Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?cited by 11
- 2023Model-agnostic Measure of Generalization Difficultycited by 7
- 2024A separability-based approach to quantifying generalization: which layer is best?cited by 5
- 2024Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Featurescited by 5
- 2024Generalization Ability of Feature-based Performance Prediction Models: A Statistical Analysis across Benchmarkscited by 4
Built on by
Citing papers per year: 2026: 2.
2 papers.