# Neural Complexity Measures

arXiv: https://arxiv.org/abs/2008.02953 · HTML: https://mlsanity.com/paper/2008.02953/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Aug 2020.

## Built on by (most cited first, 11 of 11)

- [Why neural networks find simple solutions: the many regularizers of geometric complexity](https://mlsanity.com/paper/2209.13083/) (2022)
- [Measures of Information Reflect Memorization Patterns](https://mlsanity.com/paper/2210.09404/) (2022)
- [Generalization in Neural Networks: A Broad Survey](https://mlsanity.com/paper/2209.01610/) (2022)
- [Understanding and Accelerating Neural Architecture Search with Training-Free and Theory-Grounded Metrics](https://mlsanity.com/paper/2108.11939/) (2021)
- [Leveraging PAC-Bayes Theory and Gibbs Distributions for Generalization Bounds with Complexity Measures](https://mlsanity.com/paper/2402.13285/) (2024)
- [Exploiting Explainable Metrics for Augmented SGD](https://mlsanity.com/paper/2203.16723/) (2022)
- [A classification performance evaluation measure considering data separability](https://mlsanity.com/paper/2211.05433/) (2022)
- [Reliably Detecting Model Failures in Deployment Without Labels](https://mlsanity.com/paper/2506.05047/) (2025)
- [A Ridge Too Far: Correcting Over-Shrinkage via Negative Regularization](https://mlsanity.com/paper/2508.17412/) (2025)
- [TreeNet: Layered Decision Ensembles](https://mlsanity.com/paper/2510.09654/) (2025)
- [DART: Input-Difficulty-AwaRe Adaptive Threshold for Early-Exit DNNs](https://mlsanity.com/paper/2603.12269/) (2026)

## Abstract (arXiv)

While various complexity measures for deep neural networks exist, specifying an appropriate measure capable of predicting and explaining generalization in deep networks has proven challenging. We propose Neural Complexity (NC), a meta-learning framework for predicting generalization. Our model learns a scalar complexity measure through interactions with many heterogeneous tasks in a data-driven way. The trained NC model can be added to the standard training loss to regularize any task learner in a standard supervised learning scenario. We contrast NC's approach against existing manually-designed complexity measures and other meta-learning models, and we validate NC's performance on multiple regression and classification tasks