# Characterizing Learning Dynamics of Deep Neural Networks via Complex Networks

arXiv: https://arxiv.org/abs/2110.02628 · HTML: https://mlsanity.com/paper/2110.02628/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 6 Oct 2021.

## Builds on (most cited first, 9 of 9)

- [Exact solutions to the nonlinear dynamics of learning in deep linear neural networks](https://mlsanity.com/paper/1312.6120/) (2013)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Methods for Interpreting and Understanding Deep Neural Networks](https://mlsanity.com/paper/1706.07979/) (2017)
- [Towards Stable and Efficient Training of Verifiably Robust Neural Networks](https://mlsanity.com/paper/1906.06316/) (2019)
- [Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach](https://mlsanity.com/paper/1801.10578/) (2018)
- [Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations](https://mlsanity.com/paper/1708.02691/) (2017)
- [Structure and Performance of Fully Connected Neural Networks: Emerging Complex Network Properties](https://mlsanity.com/paper/2107.14062/) (2021)
- [Deep learning systems as complex networks](https://mlsanity.com/paper/1809.10941/) (2018)
- [Emergence of Network Motifs in Deep Neural Networks](https://mlsanity.com/paper/1912.12244/) (2019)

## Built on by (most cited first, 8 of 8)

- [The Neural Race Reduction: Dynamics of Abstraction in Gated Networks](https://mlsanity.com/paper/2207.10430/) (2022)
- [Improving Deep Neural Network Random Initialization Through Neuronal Rewiring](https://mlsanity.com/paper/2207.08148/) (2022)
- [Dynamical stability and chaos in artificial neural network trajectories along training](https://mlsanity.com/paper/2404.05782/) (2024)
- [Deep Neural Networks as Complex Networks](https://mlsanity.com/paper/2209.05488/) (2022)
- [Deep Neural Networks via Complex Network Theory: a Perspective](https://mlsanity.com/paper/2404.11172/) (2024)
- [Task complexity shapes internal representations and robustness in neural networks](https://mlsanity.com/paper/2508.05463/) (2025)
- [Self-similarity Analysis in Deep Neural Networks](https://mlsanity.com/paper/2507.17785/) (2025)
- [Modeling Transformers as complex networks to analyze learning dynamics](https://mlsanity.com/paper/2509.15269/) (2025)

## Abstract (arXiv)

In this paper, we interpret Deep Neural Networks with Complex Network Theory. Complex Network Theory (CNT) represents Deep Neural Networks (DNNs) as directed weighted graphs to study them as dynamical systems. We efficiently adapt CNT measures to examine the evolution of the learning process of DNNs with different initializations and architectures: we introduce metrics for nodes/neurons and layers, namely Nodes Strength and Layers Fluctuation. Our framework distills trends in the learning dynamics and separates low from high accurate networks. We characterize populations of neural networks (ensemble analysis) and single instances (individual analysis). We tackle standard problems of image recognition, for which we show that specific learning dynamics are indistinguishable when analysed through the solely Link-Weights analysis. Further, Nodes Strength and Layers Fluctuations make unprecedented behaviours emerge: accurate networks, when compared to under-trained models, show substantially divergent distributions with the greater extremity of deviations. On top of this study, we provide an efficient implementation of the CNT metrics for both Convolutional and Fully Connected Networks, to fasten the research in this direction.