mlsanityData to 7 Oct 2026

Explaining Generalization Power of a DNN Using Interactive Concepts

arXiv 2302.13091 ↗First posted 25 Feb 2023cs.LG cs.AI cs.CVcited by 25

Not on the map. Why?

Abstract (arXiv)

This paper explains the generalization power of a deep neural network (DNN) from the perspective of interactions. Although there is no universally accepted definition of the concepts encoded by a DNN, the sparsity of interactions in a DNN has been proved, i.e., the output score of a DNN can be well explained by a small number of interactions between input variables. In this way, to some extent, we can consider such interactions as interactive concepts encoded by the DNN. Therefore, in this paper, we derive an analytic explanation of inconsistency of concepts of different complexities. This may shed new lights on using the generalization power of concepts to explain the generalization power of the entire DNN. Besides, we discover that the DNN with stronger generalization power usually learns simple concepts more quickly and encodes fewer complex concepts. We also discover the detouring dynamics of learning complex concepts, which explains both the high learning difficulty and the low generalization power of complex concepts. The code will be released when the paper is accepted.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

21 of 29 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  5. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  6. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  7. 2017Exploring Generalization in Deep Learningcited by 595
  8. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 527
  9. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  10. 2021ASAM: Adaptive Sharpness-Aware Minimization for Scale-Invariant Learning of Deep Neural Networkscited by 293
  11. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  12. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
9 more
  1. 2019Sharper bounds for uniformly stable algorithmscited by 91
  2. 2020Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithmscited by 89
  3. 2019Stiffness: A New Perspective on Generalization in Neural Networkscited by 46
  4. 2020Toward Better Generalization Bounds with Locally Elastic Stabilitycited by 41
  5. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  6. 2021Towards a Unified Information-Theoretic Framework for Generalizationcited by 29
  7. 2022From Noisy Prediction to True Label: Noisy Prediction Calibration via Generative Modelcited by 21
  8. 2021A Game-Theoretic Taxonomy of Visual Concepts in DNNscited by 16
  9. 2021A Hypothesis for the Aesthetic Appreciation in Neural Networkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 10; 2025: 6; 2026: 5.

25 papers.

Sort:

By area:

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  2. 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
  3. 2024Towards the Dynamics of a DNN Learning Symbolic Interactionscited by 13
  4. 2023Can the Inference Logic of Large Language Models be Disentangled into Symbolic Concepts?cited by 11
  5. 2023Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNscited by 10
  6. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
  7. 2023Where We Have Arrived in Proving the Emergence of Sparse Symbolic Concepts in AI Modelscited by 8
  8. 2024Towards Symbolic XAI -- Explanation Through Human Understandable Logical Relationships Between Featurescited by 7
  9. 2024Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Featurescited by 5
  10. 2024Layerwise Change of Knowledge in Neural Networkscited by 4
  11. 2024Quantifying In-Context Reasoning Effects and Memorization Effects in LLMscited by 3
  12. 2025NeurFlow: Interpreting Neural Networks through Neuron Groups and Functional Interactionscited by 3
13 more
  1. 2025InfoCons: Identifying Interpretable Critical Concepts in Point Clouds via Information Theorycited by 2
  2. 2024Why pre-training is beneficial for downstream classification tasks?cited by 1
  3. 2024Evaluating the Correctness of Inference Patterns Used by LLMs for Judgmentcited by 1
  4. 2025Revisiting Generalization Power of a DNN in Terms of Symbolic Interactionscited by 1
  5. 2025Technical Report: Quantifying and Analyzing the Generalization Power of a DNNcited by 1
  6. 2026Cross-LLM Consistency in Inference: Evidence from Shared Interactionscited by 1
  7. 2023Dissecting RGB-D Learning for Improved Multi-modal Fusioncited by 0
  8. 2024Disentangling Regional Primitives for Image Generationcited by 0
  9. 2025Conscious Gaze: Adaptive Attention Mechanisms for Hallucination Mitigation in Vision-Language Modelscited by 0
  10. 2026Hierarchical Sparse Circuit Extraction from Billion-Parameter Language Models through Scalable Attribution Graph Decompositioncited by 0
  11. 2026Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspectivecited by 0
  12. 2026A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactionscited by 0
  13. 2026Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactionscited by 0