# Training Dynamics of Deep Network Linear Regions

arXiv: https://arxiv.org/abs/2310.12977 · HTML: https://mlsanity.com/paper/2310.12977/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 42 references that are in the map.
- First posted on arXiv 19 Oct 2023.

## Builds on (most cited first, 30 of 32)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [A Closer Look at Memorization in Deep Networks](https://mlsanity.com/paper/1706.05394/) (2017)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Grokking: Generalization Beyond Overfitting on Small Algorithmic Datasets](https://mlsanity.com/paper/2201.02177/) (2022)
- [Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics](https://mlsanity.com/paper/2009.10795/) (2020)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [On the Expressive Power of Deep Neural Networks](https://mlsanity.com/paper/1606.05336/) (2016)
- [Implicit Bias of Gradient Descent on Linear Convolutional Networks](https://mlsanity.com/paper/1806.00468/) (2018)
- [The large learning rate phase of deep learning: the catapult mechanism](https://mlsanity.com/paper/2003.02218/) (2020)
- [On the Inductive Bias of Neural Tangent Kernels](https://mlsanity.com/paper/1905.12173/) (2019)
- [Complexity of Linear Regions in Deep Networks](https://mlsanity.com/paper/1901.09021/) (2019)
- [Implicit Gradient Regularization](https://mlsanity.com/paper/2009.11162/) (2020)
- [Omnigrok: Grokking Beyond Algorithmic Data](https://mlsanity.com/paper/2210.01117/) (2022)
- [Implicit Regularization in Deep Learning May Not Be Explainable by Norms](https://mlsanity.com/paper/2005.06398/) (2020)
- [Dynamics of Deep Neural Networks and Neural Tangent Hierarchy](https://mlsanity.com/paper/1909.08156/) (2019)
- [Learning in High Dimension Always Amounts to Extrapolation](https://mlsanity.com/paper/2110.09485/) (2021)
- [Data augmentation instead of explicit regularization](https://mlsanity.com/paper/1806.03852/) (2018)
- [The Geometry of Deep Networks: Power Diagram Subdivision](https://mlsanity.com/paper/1905.08443/) (2019)
- [The SSL Interplay: Augmentations, Inductive Bias, and Generalization](https://mlsanity.com/paper/2302.02774/) (2023)
- [Identity Crisis: Memorization and Generalization under Extreme Overparameterization](https://mlsanity.com/paper/1902.04698/) (2019)
- [SplineCam: Exact Visualization and Characterization of Deep Network Geometry and Decision Boundaries](https://mlsanity.com/paper/2302.12828/) (2023)
- [MaGNET: Uniform Sampling from Deep Generative Network Manifolds Without Retraining](https://mlsanity.com/paper/2110.08009/) (2021)
- [Polarity Sampling: Quality and Diversity Control of Pre-Trained Generative Networks via Singular Values](https://mlsanity.com/paper/2203.01993/) (2022)
- [A Blessing of Dimensionality in Membership Inference through Regularization](https://mlsanity.com/paper/2205.14055/) (2022)
- [Towards Democratizing Joint-Embedding Self-Supervised Learning](https://mlsanity.com/paper/2303.01986/) (2023)
- [Singular Value Perturbation and Deep Network Optimization](https://mlsanity.com/paper/2203.03099/) (2022)
- [FAIR-Ensemble: When Fairness Naturally Emerges From Deep Ensembling](https://mlsanity.com/paper/2303.00586/) (2023)
- [Implicit Regularization in Over-parameterized Neural Networks](https://mlsanity.com/paper/1903.01997/) (2019)

## Built on by (most cited first, 3 of 3)

- [Grokfast: Accelerated Grokking by Amplifying Slow Gradients](https://mlsanity.com/paper/2405.20233/) (2024)
- [Grokking in the Wild: Data Augmentation for Real-World Multi-Hop Reasoning with Transformers](https://mlsanity.com/paper/2504.20752/) (2025)
- [Computing Linear Regions in Neural Networks with Skip Connections](https://mlsanity.com/paper/2509.15441/) (2025)

## Abstract (arXiv)

The study of Deep Network (DN) training dynamics has largely focused on the evolution of the loss function, evaluated on or around train and test set data points. In fact, many DN phenomenon were first introduced in literature with that respect, e.g., double descent, grokking. In this study, we look at the training dynamics of the input space partition or linear regions formed by continuous piecewise affine DNs, e.g., networks with (leaky)ReLU nonlinearities. First, we present a novel statistic that encompasses the local complexity (LC) of the DN based on the concentration of linear regions inside arbitrary dimensional neighborhoods around data points. We observe that during training, the LC around data points undergoes a number of phases, starting with a decreasing trend after initialization, followed by an ascent and ending with a final descending trend. Using exact visualization methods, we come across the perplexing observation that during the final LC descent phase of training, linear regions migrate away from training and test samples towards the decision boundary, making the DN input-output nearly linear everywhere else. We also observe that the different LC phases are closely related to the memorization and generalization performance of the DN, especially during grokking.