# Overparametrized linear dimensionality reductions: From projection pursuit to two-layer neural networks

arXiv: https://arxiv.org/abs/2206.06526 · HTML: https://mlsanity.com/paper/2206.06526/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 36 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 3 of 3)

- [The Interpolation Phase Transition in Neural Networks: Memorization and Generalization under Lazy Training](https://mlsanity.com/paper/2007.12826/) (2020)
- [The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression](https://mlsanity.com/paper/1804.09753/) (2018)
- [Non-Gaussian Component Analysis with Log-Density Gradient Estimation](https://mlsanity.com/paper/1601.07665/) (2016)

## Built on by (most cited first, 2 of 2)

- [Wasserstein Projection Pursuit of Non-Gaussian Signals](https://mlsanity.com/paper/2302.12693/) (2023)
- [Replica Symmetry Breaking and Algorithmic Thresholds in Empirical Risk Minimization under Multi-Index Model](https://mlsanity.com/paper/2606.28573/) (2026)

## Abstract (arXiv)

Given a cloud of n data points in ℝd, consider all projections onto m-dimensional subspaces of ℝd and, for each such projection, the empirical distribution of the projected points. What does this collection of probability distributions look like when n,d grow large? We consider this question under the null model in which the points are i.i.d. standard Gaussian vectors, focusing on the asymptotic regime in which n,d→∞, with n/d→α∈ (0,∞), while m is fixed. Denoting by mathscrFm, α the set of probability distributions in ℝm that arise as low-dimensional projections in this limit, we establish new inner and outer bounds on mathscrFm, α. In particular, we characterize the Wasserstein radius of mathscrFm,α up to constant multiplicative factors, and determine it exactly for m=1. We also prove sharp bounds in terms of Kullback-Leibler divergence and R'enyi information dimension. The previous question has application to unsupervised learning methods, such as projection pursuit and independent component analysis. We introduce a version of the same problem that is relevant for supervised learning, and prove a sharp Wasserstein radius bound. As an application, we establish an upper bound on the interpolation threshold of two-layers neural networks with m hidden neurons.