# Any Target Function Exists in a Neighborhood of Any Sufficiently Wide Random Network: A Geometrical Perspective

arXiv: https://arxiv.org/abs/2001.06931 · HTML: https://mlsanity.com/paper/2001.06931/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jan 2020.

## Built on by (most cited first, 5 of 5)

- [Winning the Lottery Ahead of Time: Efficient Early Network Pruning](https://mlsanity.com/paper/2206.10451/) (2022)
- [Infinite Neural Network Quantum States: Entanglement and Training Dynamics](https://mlsanity.com/paper/2112.00723/) (2021)
- [Approximate Spectral Decomposition of Fisher Information Matrix for Simple ReLU Networks](https://mlsanity.com/paper/2111.15256/) (2021)
- [Dynamic Reinforcement Learning for Actors](https://mlsanity.com/paper/2502.10200/) (2025)
- [Deep Learning in Random Neural Fields: Numerical Experiments via Neural Tangent Kernel](https://mlsanity.com/paper/2202.05254/) (2022)

## Abstract (arXiv)

It is known that any target function is realized in a sufficiently small neighborhood of any randomly connected deep network, provided the width (the number of neurons in a layer) is sufficiently large. There are sophisticated theories and discussions concerning this striking fact, but rigorous theories are very complicated. We give an elementary geometrical proof by using a simple model for the purpose of elucidating its structure. We show that high-dimensional geometry plays a magical role: When we project a high-dimensional sphere of radius 1 to a low-dimensional subspace, the uniform distribution over the sphere reduces to a Gaussian distribution of negligibly small covariances.