# Efficient Representation of Low-Dimensional Manifolds using Deep Networks

arXiv: https://arxiv.org/abs/1602.04723 · HTML: https://mlsanity.com/paper/1602.04723/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2016.

## Built on by (most cited first, 13 of 13)

- [Topology and geometry of data manifold in deep learning](https://mlsanity.com/paper/2204.08624/) (2022)
- [Data Representations' Study of Latent Image Manifolds](https://mlsanity.com/paper/2305.19730/) (2023)
- [Topological Expressivity of ReLU Neural Networks](https://mlsanity.com/paper/2310.11130/) (2023)
- [Deep Networks Provably Classify Data on Curves](https://mlsanity.com/paper/2107.14324/) (2021)
- [Deep neural networks architectures from the perspective of manifold learning](https://mlsanity.com/paper/2306.03406/) (2023)
- [The Geometric Structure of Fully-Connected ReLU Layers](https://mlsanity.com/paper/2310.03482/) (2023)
- [Geometry of Neural Reinforcement Learning in Continuous State and Action Spaces](https://mlsanity.com/paper/2507.20853/) (2025)
- [Resource-Efficient Invariant Networks: Exponential Gains by Unrolled Optimization](https://mlsanity.com/paper/2203.05006/) (2022)
- [PointShuffleNet: Learning Non-Euclidean Features with Homotopy Equivalence and Mutual Information](https://mlsanity.com/paper/2104.02611/) (2021)
- [MD-GAN with multi-particle input: the machine learning of long-time molecular behavior from short-time MD data](https://mlsanity.com/paper/2202.00995/) (2022)
- [DeepProphet2 -- A Deep Learning Gene Recommendation Engine](https://mlsanity.com/paper/2208.01918/) (2022)
- [On the Geometry of Reinforcement Learning in Continuous State and Action Spaces](https://mlsanity.com/paper/2301.00009/) (2022)
- [The Data Manifold under the Microscope](https://mlsanity.com/paper/2606.15760/) (2026)

## Abstract (arXiv)

We consider the ability of deep neural networks to represent data that lies near a low-dimensional manifold in a high-dimensional space. We show that deep networks can efficiently extract the intrinsic, low-dimensional coordinates of such data. We first show that the first two layers of a deep network can exactly embed points lying on a monotonic chain, a special type of piecewise linear manifold, mapping them to a low-dimensional Euclidean space. Remarkably, the network can do this using an almost optimal number of parameters. We also show that this network projects nearby points onto the manifold and then embeds them with little error. We then extend these results to more general manifolds.