Efficient Representation of Low-Dimensional Manifolds using Deep Networks
Not on the map. Why?
Abstract (arXiv)
We consider the ability of deep neural networks to represent data that lies near a low-dimensional manifold in a high-dimensional space. We show that deep networks can efficiently extract the intrinsic, low-dimensional coordinates of such data. We first show that the first two layers of a deep network can exactly embed points lying on a monotonic chain, a special type of piecewise linear manifold, mapping them to a low-dimensional Euclidean space. Remarkably, the network can do this using an almost optimal number of parameters. We also show that this network projects nearby points onto the manifold and then embeds them with little error. We then extend these results to more general manifolds.
Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). First posted on arXiv 15 Feb 2016.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 2; 2022: 5; 2023: 4; 2024: 0; 2025: 1; 2026: 1.
13 papers.
Sort:
- 2022Topology and geometry of data manifold in deep learningcited by 12
- 2023Data Representations' Study of Latent Image Manifoldscited by 11
- 2023Topological Expressivity of ReLU Neural Networkscited by 9
- 2021Deep Networks Provably Classify Data on Curvescited by 6
- 2023Deep neural networks architectures from the perspective of manifold learningcited by 5
- 2023The Geometric Structure of Fully-Connected ReLU Layerscited by 4
- 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spacescited by 3
- 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimizationcited by 2
- 2021PointShuffleNet: Learning Non-Euclidean Features with Homotopy Equivalence and Mutual Informationcited by 0
- 2022MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD datacited by 0
- 2022DeepProphet2 -- A Deep Learning Gene Recommendation Enginecited by 0
- 2022On the Geometry of Reinforcement Learning in Continuous State and Action Spacescited by 0
1 more
- 2026The Data Manifold under the Microscopecited by 0
- 2026The Data Manifold under the Microscope14 Jun 2026
- 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces28 Jul 2025
- 2023Topological Expressivity of ReLU Neural Networks17 Oct 2023
- 2023The Geometric Structure of Fully-Connected ReLU Layers5 Oct 2023
- 2023Deep neural networks architectures from the perspective of manifold learning6 Jun 2023
- 2023Data Representations' Study of Latent Image Manifolds31 May 2023
- 2022On the Geometry of Reinforcement Learning in Continuous State and Action Spaces29 Dec 2022
- 2022DeepProphet2 -- A Deep Learning Gene Recommendation Engine3 Aug 2022
- 2022Topology and geometry of data manifold in deep learning19 Apr 2022
- 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization9 Mar 2022
- 2022MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD data2 Feb 2022
- 2021Deep Networks Provably Classify Data on Curves29 Jul 2021
- 2022Topology and geometry of data manifold in deep learningcited by 12
- 2023Data Representations' Study of Latent Image Manifoldscited by 11
- 2023Topological Expressivity of ReLU Neural Networkscited by 9
- 2021Deep Networks Provably Classify Data on Curvescited by 6
- 2023Deep neural networks architectures from the perspective of manifold learningcited by 5
- 2023The Geometric Structure of Fully-Connected ReLU Layerscited by 4
- 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spacescited by 3
- 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimizationcited by 2
- 2021PointShuffleNet: Learning Non-Euclidean Features with Homotopy Equivalence and Mutual Informationcited by 0
- 2022MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD datacited by 0
- 2022DeepProphet2 -- A Deep Learning Gene Recommendation Enginecited by 0
- 2022On the Geometry of Reinforcement Learning in Continuous State and Action Spacescited by 0
12 most cited of 13 All 13 (JSON)
- 2026The Data Manifold under the Microscope14 Jun 2026
- 2025Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces28 Jul 2025
- 2023Topological Expressivity of ReLU Neural Networks17 Oct 2023
- 2023The Geometric Structure of Fully-Connected ReLU Layers5 Oct 2023
- 2023Deep neural networks architectures from the perspective of manifold learning6 Jun 2023
- 2023Data Representations' Study of Latent Image Manifolds31 May 2023
- 2022On the Geometry of Reinforcement Learning in Continuous State and Action Spaces29 Dec 2022
- 2022DeepProphet2 -- A Deep Learning Gene Recommendation Engine3 Aug 2022
- 2022Topology and geometry of data manifold in deep learning19 Apr 2022
- 2022Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization9 Mar 2022
- 2022MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD data2 Feb 2022
- 2021Deep Networks Provably Classify Data on Curves29 Jul 2021
12 newest of 13 All 13 (JSON)