mlsanityData to 7 Oct 2026

Towards the Dynamics of a DNN Learning Symbolic Interactions

arXiv 2407.19198 ↗First posted 27 Jul 2024cs.LG cs.AI cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

This study proves the two-phase dynamics of a deep neural network (DNN) learning interactions. Despite the long disappointing view of the faithfulness of post-hoc explanation of a DNN, a series of theorems have been proven in recent years to show that for a given input sample, a small set of interactions between input variables can be considered as primitive inference patterns that faithfully represent a DNN's detailed inference logic on that sample. Particularly, Zhang et al. have observed that various DNNs all learn interactions of different complexities in two distinct phases, and this two-phase dynamics well explains how a DNN changes from under-fitting to over-fitting. Therefore, in this study, we mathematically prove the two-phase dynamics of interactions, providing a theoretical mechanism for how the generalization power of a DNN changes during the training process. Experiments show that our theory well predicts the real dynamics of interactions on different DNNs trained for various tasks.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 27 of its 46 references that are in the map. First posted on arXiv 27 Jul 2024.

Builds on

27 of 46 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  6. 2018Dynamic Graph CNN for Learning on Point Cloudscited by 2,220
  7. 2017Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)cited by 1,309
  8. 2018Sanity Checks for Saliency Mapscited by 1,149
  9. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  10. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  11. 2017Distilling a Neural Network Into a Soft Decision Treecited by 230
  12. 2019The Shapley Taylor Interaction Indexcited by 143
15 more
  1. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  2. 2021Discovering and Explaining the Representation Bottleneck of DNNscited by 42
  3. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  4. 2020Interpreting and Boosting Dropout from a Game-Theoretic Viewcited by 34
  5. 2018Explainable Neural Networks based on Additive Index Modelscited by 33
  6. 2023Does a Neural Network Really Encode Symbolic Concepts?cited by 27
  7. 2023Explaining Generalization Power of a DNN Using Interactive Conceptscited by 25
  8. 2021A Game-Theoretic Taxonomy of Visual Concepts in DNNscited by 16
  9. 2024Defining and Extracting generalizable interaction primitives from DNNscited by 14
  10. 2022Proving Common Mechanisms Shared by Twelve Methods of Boosting Adversarial Transferabilitycited by 12
  11. 2018Considerations When Learning Additive Explanations for Black-Box Modelscited by 10
  12. 2023Technical Note: Defining and Quantifying AND-OR Interactions for Faithful and Concise Explanation of DNNscited by 10
  13. 2021A Hypothesis for the Aesthetic Appreciation in Neural Networkscited by 9
  14. 2023Explaining How a Neural Network Play the Go Game and Let People Learncited by 6
  15. 2024Two-Phase Dynamics of Interactions Explains the Starting Point of a DNN Learning Over-Fitted Featurescited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 5; 2026: 5.

13 papers.

Sort:

By area:

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  2. 2025ProxySPEX: Inference-Efficient Interpretability via Sparse Feature Interactions in LLMscited by 16
  3. 2024Tracking the Feature Dynamics in LLM Training: A Mechanistic Studycited by 15
  4. 2025SPEX: Scaling Feature Interaction Explanations for LLMscited by 12
  5. 2025Physics of Skill Learningcited by 8
  6. 2025Revisiting Generalization Power of a DNN in Terms of Symbolic Interactionscited by 1
  7. 2025Technical Report: Quantifying and Analyzing the Generalization Power of a DNNcited by 1
  8. 2026textitFocaLogic: Logic-Based Interpretation of Visual Model Decisionscited by 1
  9. 2026Cross-LLM Consistency in Inference: Evidence from Shared Interactionscited by 1
  10. 2024Disentangling Regional Primitives for Image Generationcited by 0
  11. 2026Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspectivecited by 0
  12. 2026A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactionscited by 0
1 more
  1. 2026Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactionscited by 0