# Power-law Scaling to Assist with Key Challenges in Artificial Intelligence

arXiv: https://arxiv.org/abs/2211.08430 · HTML: https://mlsanity.com/paper/2211.08430/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 35 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Builds on (most cited first, 9 of 9)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Deep Learning Scaling is Predictable, Empirically](https://mlsanity.com/paper/1712.00409/) (2017)
- [A Constructive Prediction of the Generalization Error Across Scales](https://mlsanity.com/paper/1909.12673/) (2019)
- [Discovering Phase Transitions with Unsupervised Learning](https://mlsanity.com/paper/1606.00318/) (2016)
- [Low Photon Count Phase Retrieval Using Deep Learning](https://mlsanity.com/paper/1806.10029/) (2018)
- [Enabling real-time multi-messenger astrophysics discoveries with deep learning](https://mlsanity.com/paper/1911.11779/) (2019)
- [Brain experiments imply adaptation mechanisms which outperform common AI learning algorithms](https://mlsanity.com/paper/2005.04106/) (2020)
- [Putting a bug in ML: The moth olfactory network learns to read MNIST](https://mlsanity.com/paper/1802.05405/) (2018)
- [Optimal Network Topology for Effective Collective Response](https://mlsanity.com/paper/1807.04631/) (2018)

## Built on by (most cited first, 6 of 6)

- [Dense Hebbian neural networks: a replica symmetric picture of supervised learning](https://mlsanity.com/paper/2212.00606/) (2022)
- [Guided Transfer Learning](https://mlsanity.com/paper/2303.16154/) (2023)
- [Learning Mechanism Underlying NLP Pre-Training and Fine-Tuning](https://mlsanity.com/paper/2509.03407/) (2025)
- [Statistical Mechanics of Learning via Reverberation in Bidirectional Associative Memories](https://mlsanity.com/paper/2307.08365/) (2023)
- [Role of Delay in Brain Dynamics](https://mlsanity.com/paper/2410.11384/) (2024)
- [Neural Scaling Laws for Deep Regression](https://mlsanity.com/paper/2509.10000/) (2025)

## Abstract (arXiv)

Power-law scaling, a central concept in critical phenomena, is found to be useful in deep learning, where optimized test errors on handwritten digit examples converge as a power-law to zero with database size. For rapid decision making with one training epoch, each example is presented only once to the trained network, the power-law exponent increased with the number of hidden layers. For the largest dataset, the obtained test error was estimated to be in the proximity of state-of-the-art algorithms for large epoch numbers. Power-law scaling assists with key challenges found in current artificial intelligence applications and facilitates an a priori dataset size estimation to achieve a desired test accuracy. It establishes a benchmark for measuring training complexity and a quantitative hierarchy of machine learning tasks and algorithms.