# Isometric Autoencoders

arXiv: https://arxiv.org/abs/2006.09289 · HTML: https://mlsanity.com/paper/2006.09289/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jun 2020.

## Built on by (most cited first, 18 of 18)

- [The Dimpled Manifold Model of Adversarial Examples in Machine Learning](https://mlsanity.com/paper/2106.10151/) (2021)
- [Geometric Autoencoders -- What You See is What You Decode](https://mlsanity.com/paper/2306.17638/) (2023)
- [Convolutional Normalization: Improving Deep Convolutional Network Robustness and Training](https://mlsanity.com/paper/2103.00673/) (2021)
- [Nonlinear Isometric Manifold Learning for Injective Normalizing Flows](https://mlsanity.com/paper/2203.03934/) (2022)
- [Geometry-Preserving Encoder/Decoder in Latent Generative Models](https://mlsanity.com/paper/2501.09876/) (2025)
- [Principal Manifold Flows](https://mlsanity.com/paper/2202.07037/) (2022)
- [Convergent autoencoder approximation of low bending and low distortion manifold embeddings](https://mlsanity.com/paper/2208.10193/) (2022)
- [Monotone Generative Modeling via a Gromov-Monge Embedding](https://mlsanity.com/paper/2311.01375/) (2023)
- [Disentangled Representation Learning with the Gromov-Monge Gap](https://mlsanity.com/paper/2407.07829/) (2024)
- [Unsupervised discovery of the shared and private geometry in multi-view data](https://mlsanity.com/paper/2408.12091/) (2024)
- [Multi-Scale Geometric Autoencoder](https://mlsanity.com/paper/2509.24168/) (2025)
- [Bi-Lipschitz Autoencoder With Injectivity Guarantee](https://mlsanity.com/paper/2604.06701/) (2026)
- [Learning low bending and low distortion manifold embeddings](https://mlsanity.com/paper/2104.13189/) (2021)
- [Assessing local deformation and computing scalar curvature with nonlinear conformal regularization of decoders](https://mlsanity.com/paper/2508.20413/) (2025)
- [Twincher: Bijective Representation Learning for Robust Inversion of Continuous Systems](https://mlsanity.com/paper/2605.13470/) (2026)
- [Beyond Uniform Local Isometry and Topology: FactoMap for Disentangled Representations](https://mlsanity.com/paper/2608.24762/) (2026)
- [Partially Linear Autoencoders for Manifold Learning and Dimensionality Reduction](https://mlsanity.com/paper/2608.29867/) (2026)
- [A shape-similarity latent space for fluid interfaces: invertible reduced-order modelling of droplet morphology](https://mlsanity.com/paper/2609.37947/) (2026)

## Abstract (arXiv)

High dimensional data is often assumed to be concentrated on or near a low-dimensional manifold. Autoencoders (AE) is a popular technique to learn representations of such data by pushing it through a neural network with a low dimension bottleneck while minimizing a reconstruction error. Using high capacity AE often leads to a large collection of minimizers, many of which represent a low dimensional manifold that fits the data well but generalizes poorly. Two sources of bad generalization are: extrinsic, where the learned manifold possesses extraneous parts that are far from the data; and intrinsic, where the encoder and decoder introduce arbitrary distortion in the low dimensional parameterization. An approach taken to alleviate these issues is to add a regularizer that favors a particular solution; common regularizers promote sparsity, small derivatives, or robustness to noise. In this paper, we advocate an isometry (i.e., local distance preserving) regularizer. Specifically, our regularizer encourages: (i) the decoder to be an isometry; and (ii) the encoder to be the decoder's pseudo-inverse, that is, the encoder extends the inverse of the decoder to the ambient space by orthogonal projection. In a nutshell, (i) and (ii) fix both intrinsic and extrinsic degrees of freedom and provide a non-linear generalization to principal component analysis (PCA). Experimenting with the isometry regularizer on dimensionality reduction tasks produces useful low-dimensional data representations.