mlsanityData to 14 Oct 2026

Random Matrix Theory Proves that Deep Learning Representations of GAN-data Behave as Gaussian Mixtures

arXiv 2001.08370 ↗First posted 21 Jan 2020cs.LG stat.MLcited by 47

Not on the map. Why?

Abstract (arXiv)

This paper shows that deep learning (DL) representations of data produced by generative adversarial nets (GANs) are random vectors which fall within the class of so-called textitconcentrated random vectors. Further exploiting the fact that Gram matrices, of the type G = XT X with X=[x1,ldots,xn]∈ ℝp× n and xi independent concentrated random vectors from a mixture model, behave asymptotically (as n,p→ ∞) as if the xi were drawn from a Gaussian mixture, suggests that DL representations of GAN-data can be fully described by their first two statistical moments for a wide range of standard classifiers. Our theoretical findings are validated by generating images with the BigGAN model and across different popular deep representation networks.

Key facts. Cited by 47 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Jan 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 7; 2022: 5; 2023: 9; 2024: 11; 2025: 11; 2026: 4.

47 papers.

Sort:

By area:

  1. 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
  2. 2021Learning Gaussian Mixtures with Generalised Linear Models: Precise Asymptotics in High-dimensionscited by 54
  3. 2024Gramian Multimodal Representation Learning and Alignmentcited by 54
  4. 2022Memorization and Optimization in Deep Neural Networks with Minimum Over-parameterizationcited by 36
  5. 2023Universality laws for Gaussian mixtures in generalized linear modelscited by 28
  6. 2022Gaussian Universality of Perceptrons with Random Labelscited by 26
  7. 2023Precise Asymptotic Analysis of Deep Random Feature Modelscited by 22
  8. 2022Theoretical characterization of uncertainty in high-dimensional linear classificationcited by 18
  9. 2023Beyond the Universal Law of Robustness: Sharper Laws for Random Features and Neural Tangent Kernelscited by 13
  10. 2023PAC-Bayesian Generalization Bounds for Adversarial Generative Modelscited by 13
  11. 2022Decomposing neural networks as mappings of correlation functionscited by 12
  12. 2023Classification of Heavy-tailed Features in High Dimensions: a Superstatistical Approachcited by 11
35 more
  1. 2023How Spurious Features Are Memorized: Precise Analysis for Random and NTK Featurescited by 11
  2. 2023The Underlying Scaling Laws and Universal Statistical Structure of Complex Datasetscited by 9
  3. 2025Double descent in quantum kernel methodscited by 9
  4. 2024SPQR: Controlling Q-ensemble Independence with Spiked Random Model for Reinforcement Learningcited by 8
  5. 2024Analysing Multi-Task Regression via Random Matrix Theory with Application to Time Series Forecastingcited by 8
  6. 2021Federated Learning-based Active Authentication on Mobile Devicescited by 7
  7. 2021Random matrices in service of ML footprint: ternary random features with no performance losscited by 7
  8. 2022Overparameterized random feature regression with nearly orthogonal datacited by 7
  9. 2023On progressive sharpening, flat minima and generalisationcited by 7
  10. 2023Random Matrix Analysis to Balance between Supervised and Unsupervised Learning under the Low Density Separation Assumptioncited by 6
  11. 2024Deep Equilibrium Models are Almost Equivalent to Not-so-deep Explicit Models for High-dimensional Gaussian Mixturescited by 6
  12. 2021Node Feature Kernels Increase Graph Convolutional Network Robustnesscited by 5
  13. 2024Towards Understanding the Word Sensitivity of Attention Layers: A Study via Random Featurescited by 5
  14. 2021PCA-based Multi Task Learning: a Random Matrix Approachcited by 4
  15. 2024A Universal Metric of Dataset Similarity for Cross-silo Federated Learningcited by 4
  16. 2025Asymptotic Analysis of Two-Layer Neural Networks after One Gradient Step under Gaussian Mixtures Data with Structurecited by 4
  17. 2024"Lossless" Compression of Deep Neural Networks: A High-dimensional Neural Tangent Kernel Approachcited by 3
  18. 2025High-Dimensional Analysis of Bootstrap Ensemble Classifierscited by 3
  19. 2021On some theoretical limitations of Generative Adversarial Networkscited by 2
  20. 2024Deep Neural Network Models Trained With A Fixed Random Classifier Transfer Better Across Domainscited by 2
  21. 2024High-dimensional Learning with Noisy Labelscited by 2
  22. 2025Spurious Correlation-Aware Embedding Regularization for Worst-Group Robustnesscited by 2
  23. 2024A Large Dimensional Analysis of Multi-task Semi-Supervised Learningcited by 1
  24. 2025Incorporating priors in learning: a random matrix study under a teacher-student frameworkcited by 1
  25. 2025Concept activation vectors: a unifying view and adversarial attackscited by 1
  26. 2026Characterization of Gaussian Universality Breakdown in High-Dimensional Empirical Risk Minimizationcited by 1
  27. 2024Classifying Overlapping Gaussian Mixtures in High Dimensions: From Optimal Classifiers to Neural Netscited by 0
  28. 2025Gaussian Universality for Diffusion Modelscited by 0
  29. 2025Meta-Learning Transformers to Improve In-Context Generalizationcited by 0
  30. 2025A Random Matrix Perspective of Echo State Networks: From Precise Bias--Variance Characterization to Optimal Regularizationcited by 0
  31. 2025One-Bit Quantization for Random Features Modelscited by 0
  32. 2025Generalization in Representation Models via Random Matrix Theory: Application to Recurrent Networkscited by 0
  33. 2026How Does Attention Help? Insights from Random Matrices on Signal Recovery from Sequence Modelscited by 0
  34. 2026α-TCAV: A Unified Framework for Testing with Concept Activation Vectorscited by 0
  35. 2026Cautious optimism for deep parameterized quantum circuitscited by 0