# Convergent autoencoder approximation of low bending and low distortion manifold embeddings

arXiv: https://arxiv.org/abs/2208.10193 · HTML: https://mlsanity.com/paper/2208.10193/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 22 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Accelerating 3D Deep Learning with PyTorch3D](https://mlsanity.com/paper/2007.08501/) (2020)
- [Error bounds for approximations with deep ReLU neural networks in Ws,p norms](https://mlsanity.com/paper/1902.07896/) (2019)
- [Optimal Approximation with Sparsely Connected Deep Neural Networks](https://mlsanity.com/paper/1705.01714/) (2017)
- [Understanding and Improving Interpolation in Autoencoders via an Adversarial Regularizer](https://mlsanity.com/paper/1807.07543/) (2018)
- [The Riemannian Geometry of Deep Generative Models](https://mlsanity.com/paper/1711.08014/) (2017)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [LOCA: LOcal Conformal Autoencoder for standardized data coordinates](https://mlsanity.com/paper/2004.07234/) (2020)
- [Isometric Autoencoders](https://mlsanity.com/paper/2006.09289/) (2020)
- [DIMAL: Deep Isometric Manifold Learning Using Sparse Geodesic Sampling](https://mlsanity.com/paper/1711.06011/) (2017)
- [Intrinsic Isometric Manifold Learning with Application to Localization](https://mlsanity.com/paper/1806.00556/) (2018)
- [LDLE: Low Distortion Local Eigenmaps](https://mlsanity.com/paper/2101.11055/) (2021)

## Built on by (most cited first, 4 of 4)

- [Geometry-Preserving Encoder/Decoder in Latent Generative Models](https://mlsanity.com/paper/2501.09876/) (2025)
- [Distributional Autoencoders Know the Score](https://mlsanity.com/paper/2502.11583/) (2025)
- [Geodesic Calculus on Implicitly Defined Latent Manifolds](https://mlsanity.com/paper/2510.09468/) (2025)
- [Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction](https://mlsanity.com/paper/2608.29867/) (2026)

## Abstract (arXiv)

Autoencoders, which consist of an encoder and a decoder, are widely used in machine learning for dimension reduction of high-dimensional data. The encoder embeds the input data manifold into a lower-dimensional latent space, while the decoder represents the inverse map, providing a parametrization of the data manifold by the manifold in latent space. A good regularity and structure of the embedded manifold may substantially simplify further data processing tasks such as cluster analysis or data interpolation. We propose and analyze a novel regularization for learning the encoder component of an autoencoder: a loss functional that prefers isometric, extrinsically flat embeddings and allows to train the encoder on its own. To perform the training it is assumed that for pairs of nearby points on the input manifold their local Riemannian distance and their local Riemannian average can be evaluated. The loss functional is computed via Monte Carlo integration with different sampling strategies for pairs of points on the input manifold. Our main theorem identifies a geometric loss functional of the embedding map as the Gamma-limit of the sampling-dependent loss functionals. Numerical tests, using image data that encodes different explicitly given data manifolds, show that smooth manifold embeddings into latent space are obtained. Due to the promotion of extrinsic flatness, these embeddings are regular enough such that interpolation between not too distant points on the manifold is well approximated by linear interpolation in latent space as one possible postprocessing.