mlsanityData to 7 Oct 2026

The Intrinsic Dimension of Images and Its Impact on Learning

arXiv 2104.08894 ↗First posted 18 Apr 2021cs.CV cs.LG stat.MLcited by 307

Not on the map. Why?

Cited by 70 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

It is widely believed that natural image data exhibits low-dimensional structure despite the high dimensionality of conventional pixel representations. This idea underlies a common intuition for the remarkable success of deep learning in computer vision. In this work, we apply dimension estimation tools to popular datasets and investigate the role of low-dimensional structure in deep learning. We find that common natural image datasets indeed have very low intrinsic dimension relative to the high number of pixels in the images. Additionally, we find that low dimensional datasets are easier for neural networks to learn, and models solving these tasks generalize better from training to test data. Along the way, we develop a technique for validating our dimension estimation tools on synthetic data generated by GANs allowing us to actively manipulate the intrinsic dimension by controlling the image generation process. Code for our experiments may be found here https://github.com/ppope/dimensions.

Key facts. Cited by 307 papers in the map (data through 7 Oct 2026). Builds on 16 of its 48 references that are in the map. First posted on arXiv 18 Apr 2021.

Builds on

16 of 48 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2019Intrinsic dimension of data representations in deep neural networkscited by 276
  7. 2019Approximating CNNs with Bag-of-local-Features models works surprisingly well on ImageNetcited by 260
  8. 2018Estimating the intrinsic dimension of datasets by a minimal neighborhood informationcited by 246
  9. 2018Disentangling Adversarial Robustness and Generalizationcited by 171
  10. 2018On the Intrinsic Dimensionality of Image Representationscited by 57
  11. 2018Deep convolutional recurrent autoencoders for learning low-dimensional feature dynamics of fluid systemscited by 54
  12. 2019Understanding Generalization through Visualizationscited by 54
4 more
  1. 2017LDMNet: Low Dimensional Manifold Regularized Neural Networkscited by 21
  2. 2019Geometry-Aware Maximum Likelihood Estimation of Intrinsic Dimensioncited by 15
  3. 2020Texture Interpolation for Probing Visual Perceptioncited by 15
  4. 2016Minimax Rates for Estimating the Dimension of a Manifoldcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 6; 2022: 42; 2023: 50; 2024: 66; 2025: 68; 2026: 75.

307 papers.

Sort:

By area:

  1. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  2. 2024Guiding a Diffusion Model with a Bad Version of Itselfcited by 286
  3. 2025Diffusion Transformers with Representation Autoencoderscited by 275
  4. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Datacited by 171
  5. 2021A Universal Law of Robustness via Isoperimetrycited by 150
  6. 2022Score-Based Generative Models Detect Manifoldscited by 121
  7. 2023Intrinsic Dimension Estimation for Robust Detection of AI-Generated Textscited by 111
  8. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 85
  9. 2024Better than classical? The subtle art of benchmarking quantum machine learning modelscited by 82
  10. 2023How Deep Neural Networks Learn Compositional Data: The Random Hierarchy Modelcited by 70
  11. 2022Verifying the Union of Manifolds Hypothesis for Image Datacited by 62
  12. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
38 more
  1. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Modelscited by 56
  2. 2024Emergence of a High-Dimensional Abstraction Phase in Language Transformerscited by 56
  3. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributionscited by 53
  4. 2022Neural networks trained with SGD learn distributions of increasing complexitycited by 50
  5. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  6. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptionscited by 48
  7. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  8. 2024Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimensioncited by 45
  9. 2023Detecting Images Generated by Deep Diffusion Models using their Local Intrinsic Dimensionalitycited by 44
  10. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionalitycited by 44
  11. 2024Linear Convergence of Diffusion Models Under the Manifold Hypothesiscited by 42
  12. 2021The Dimpled Manifold Model of Adversarial Examples in Machine Learningcited by 41
  13. 2022Can Push-forward Generative Models Fit Multimodal Distributions?cited by 41
  14. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 41
  15. 2024On Statistical Rates and Provably Efficient Criteria of Latent Diffusion Transformers (DiTs)cited by 39
  16. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  17. 2022Your diffusion model secretly knows the dimension of the data manifoldcited by 35
  18. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Modelscited by 35
  19. 2023Predicting Grokking Long Before it Happens: A look into the loss landscape of models which grokcited by 31
  20. 2022LIDL: Local Intrinsic Dimension Estimation Using Approximate Likelihoodcited by 30
  21. 2024Understanding Scaling Laws with Statistical and Approximation Theory for Transformer Neural Networks on Intrinsically Low-dimensional Datacited by 30
  22. 2023On the Embedding Collapse when Scaling up Recommendation Modelscited by 29
  23. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  24. 2022CaloMan: Fast generation of calorimeter showers with density estimation on learned manifoldscited by 28
  25. 2022Predicting is not Understanding: Recognizing and Addressing Underspecification in Machine Learningcited by 26
  26. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  27. 2023Bridging Information-Theoretic and Geometric Compression in Language Modelscited by 26
  28. 2024Nonlinear spiked covariance matrices and signal propagation in deep neural networkscited by 25
  29. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  30. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  31. 2023Learning Manifold Dimensions with Conditional Variational Autoencoderscited by 20
  32. 2024Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapescited by 20
  33. 2025Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptivecited by 20
  34. 2023Interpreting and Improving Diffusion Models from an Optimization Perspectivecited by 19
  35. 2023Complexity Matters: Rethinking the Latent Space for Generative Modelingcited by 19
  36. 2024Hardness of Learning Neural Networks under the Manifold Hypothesiscited by 18
  37. 2025Understanding Representation Dynamics of Diffusion Models via Low-Dimensional Modelingcited by 18
  38. 2025The Intrinsic Dimension of Prompts in Internal Representations of Large Language Modelscited by 17

50 most cited of 307 All 307 (JSON)