# Geometric Understanding of Deep Learning

arXiv: https://arxiv.org/abs/1805.10451 · HTML: https://mlsanity.com/paper/1805.10451/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 May 2018.

## Built on by (most cited first, 30 of 41)

- [Correcting Biased Centered Kernel Alignment Measures in Biological and Artificial Neural Networks](https://mlsanity.com/paper/2405.01012/) (2024)
- [FocusNetv2: Imbalanced Large and Small Organ Segmentation with Adversarial Shape Constraint for Head and Neck CT Images](https://mlsanity.com/paper/2104.01771/) (2021)
- [DPM-OT: A New Diffusion Probabilistic Model Based on Optimal Transport](https://mlsanity.com/paper/2307.11308/) (2023)
- [Predicting and Enhancing the Fairness of DNNs with the Curvature of Perceptual Manifolds](https://mlsanity.com/paper/2303.12307/) (2023)
- [A Unified Interactive Model Evaluation for Classification, Object Detection, and Instance Segmentation in Computer Vision](https://mlsanity.com/paper/2308.05168/) (2023)
- [Intuitive Shape Editing in Latent Space](https://mlsanity.com/paper/2111.12488/) (2021)
- [Unveiling and Mitigating Generalized Biases of DNNs through the Intrinsic Dimensions of Perceptual Manifolds](https://mlsanity.com/paper/2404.13859/) (2024)
- [Topological Deep Learning: Classification Neural Networks](https://mlsanity.com/paper/2102.08354/) (2021)
- [NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension](https://mlsanity.com/paper/2211.12759/) (2022)
- [Spatial and Modal Optimal Transport for Fast Cross-Modal MRI Reconstruction](https://mlsanity.com/paper/2305.02774/) (2023)
- [Orthogonal Uncertainty Representation of Data Manifold for Robust Long-Tailed Learning](https://mlsanity.com/paper/2310.10090/) (2023)
- [Aligning Logits Generatively for Principled Black-Box Knowledge Distillation](https://mlsanity.com/paper/2205.10490/) (2022)
- [Priors in Deep Image Restoration and Enhancement: A Survey](https://mlsanity.com/paper/2206.02070/) (2022)
- [A Deep Double Ritz Method (D2RM) for solving Partial Differential Equations using Neural Networks](https://mlsanity.com/paper/2211.03627/) (2022)
- [Histopathology Image Classification using Deep Manifold Contrastive Learning](https://mlsanity.com/paper/2306.14459/) (2023)
- [A Local Geometric Interpretation of Feature Extraction in Deep Feedforward Neural Networks](https://mlsanity.com/paper/2202.04632/) (2022)
- [OT-Net: A Reusable Neural Optimal Transport Solver](https://mlsanity.com/paper/2306.08233/) (2023)
- [NoPain: No-box Point Cloud Attack via Optimal Transport Singular Boundary](https://mlsanity.com/paper/2503.00063/) (2025)
- [Convex Smoothed Autoencoder-Optimal Transport model](https://mlsanity.com/paper/2101.05679/) (2021)
- [How to Explain Neural Networks: an Approximation Perspective](https://mlsanity.com/paper/2105.07831/) (2021)
- [A Robust Classification-autoencoder to Defend Outliers and Adversaries](https://mlsanity.com/paper/2106.15927/) (2021)
- [Detecting danger in gridworlds using Gromov's Link Condition](https://mlsanity.com/paper/2201.06274/) (2022)
- [On the Relationship Between Adversarial Robustness and Decision Region in Deep Neural Networks](https://mlsanity.com/paper/2207.03400/) (2022)
- [Data-Centric Long-Tailed Image Recognition](https://mlsanity.com/paper/2311.01744/) (2023)
- [A mathematical framework of intelligence and consciousness based on Riemannian Geometry](https://mlsanity.com/paper/2407.11024/) (2024)
- [HOTS3D: Hyper-Spherical Optimal Transport for Semantic Alignment of Text-to-3D Generation](https://mlsanity.com/paper/2407.14419/) (2024)
- [Low-Rank Tensor Recovery via Variational Schatten-p Quasi-Norm and Jacobian Regularization](https://mlsanity.com/paper/2506.22134/) (2025)
- [Learning Low-dimensional Manifolds for Scoring of Tissue Microarray Images](https://mlsanity.com/paper/2102.11396/) (2021)
- [Slope and generalization properties of neural networks](https://mlsanity.com/paper/2107.01473/) (2021)
- [ETO Meets Scheduling: Learning Key Knowledge from Single-Objective Problems to Multi-Objective Problem](https://mlsanity.com/paper/2206.12902/) (2022)

## Abstract (arXiv)

Deep learning is the mainstream technique for many machine learning tasks, including image recognition, machine translation, speech recognition, and so on. It has outperformed conventional methods in various fields and achieved great successes. Unfortunately, the understanding on how it works remains unclear. It has the central importance to lay down the theoretic foundation for deep learning. In this work, we give a geometric view to understand deep learning: we show that the fundamental principle attributing to the success is the manifold structure in data, namely natural high dimensional data concentrates close to a low-dimensional manifold, deep learning learns the manifold and the probability distribution on it. We further introduce the concepts of rectified linear complexity for deep neural network measuring its learning capability, rectified linear complexity of an embedding manifold describing the difficulty to be learned. Then we show for any deep neural network with fixed architecture, there exists a manifold that cannot be learned by the network. Finally, we propose to apply optimal mass transportation theory to control the probability distribution in the latent space.