# Geometric Autoencoders -- What You See is What You Decode

arXiv: https://arxiv.org/abs/2306.17638 · HTML: https://mlsanity.com/paper/2306.17638/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 56 references that are in the map.
- First posted on arXiv 30 Jun 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013)
- [From Variational to Deterministic Autoencoders](https://mlsanity.com/paper/1903.12436/) (2019)
- [Topological Autoencoders](https://mlsanity.com/paper/1906.00722/) (2019)
- [Parametric UMAP embeddings for representation and semi-supervised learning](https://mlsanity.com/paper/2009.12981/) (2020)
- [geomstats: a Python Package for Riemannian Geometry in Machine Learning](https://mlsanity.com/paper/1805.08308/) (2018)
- [Metrics for Deep Generative Models](https://mlsanity.com/paper/1711.01204/) (2017)
- [von Mises-Fisher Loss: An Exploration of Embedding Geometries for Supervised Learning](https://mlsanity.com/paper/2103.15718/) (2021)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [Learning Flat Latent Manifolds with VAEs](https://mlsanity.com/paper/2002.04881/) (2020)
- [On UMAP's true loss function](https://mlsanity.com/paper/2103.14608/) (2021)
- [Learning Topology-Preserving Data Representations](https://mlsanity.com/paper/2302.00136/) (2023)
- [Heavy-tailed kernels reveal a finer cluster structure in t-SNE visualisations](https://mlsanity.com/paper/1902.05804/) (2019)
- [Isometric Autoencoders](https://mlsanity.com/paper/2006.09289/) (2020)
- [Unsupervised visualization of image datasets using contrastive learning](https://mlsanity.com/paper/2210.09879/) (2022)
- [Markov-Lipschitz Deep Learning](https://mlsanity.com/paper/2006.08256/) (2020)

## Built on by (most cited first, 23 of 23)

- [Metric Flow Matching for Smooth Interpolations on the Data Manifold](https://mlsanity.com/paper/2405.14780/) (2024)
- [Temporal Distance-aware Transition Augmentation for Offline Model-based Reinforcement Learning](https://mlsanity.com/paper/2505.13144/) (2025)
- [MMP++: Motion Manifold Primitives with Parametric Curve Models](https://mlsanity.com/paper/2310.17072/) (2023)
- [Enforcing Latent Euclidean Geometry in Single-Cell VAEs for Manifold Interpolation](https://mlsanity.com/paper/2507.11789/) (2025)
- [Random Forest Autoencoders for Guided Representation Learning](https://mlsanity.com/paper/2502.13257/) (2025)
- [DREAMS: Preserving both Local and Global Structure in Dimensionality Reduction](https://mlsanity.com/paper/2508.13747/) (2025)
- [Navigating the Effect of Parametrization for Dimensionality Reduction](https://mlsanity.com/paper/2411.15894/) (2024)
- [Improving Feasibility via Fast Autoencoder-Based Projections](https://mlsanity.com/paper/2604.03489/) (2026)
- [Structure-preserving contrastive learning for spatial time series](https://mlsanity.com/paper/2502.06380/) (2025)
- [Latent Manifold Reconstruction and Representation with Topological and Geometrical Regularization](https://mlsanity.com/paper/2505.04412/) (2025)
- [Geometric Autoencoder Priors for Bayesian Inversion: Learn First Observe Later](https://mlsanity.com/paper/2509.19929/) (2025)
- [Multi-Scale Geometric Autoencoder](https://mlsanity.com/paper/2509.24168/) (2025)
- [On Geometry Regularization in Autoencoder Reduced-Order Models with Latent Neural ODE Dynamics](https://mlsanity.com/paper/2603.03238/) (2026)
- [Bi-Lipschitz Autoencoder With Injectivity Guarantee](https://mlsanity.com/paper/2604.06701/) (2026)
- [Integrative Variational Autoencoders for Generative Modeling of an Image Outcome with Multiple Input Images](https://mlsanity.com/paper/2402.02734/) (2024)
- [Proper Latent Decomposition](https://mlsanity.com/paper/2412.00785/) (2024)
- [Investigating Image Manifolds of 3D Objects: Learning, Shape Analysis, and Comparisons](https://mlsanity.com/paper/2503.06773/) (2025)
- [Nonlinear Model Order Reduction of Dynamical Systems in Process Engineering: Review and Comparison](https://mlsanity.com/paper/2506.12819/) (2025)
- [Multivariate Time Series Forecasting with Hybrid Euclidean-SPD Manifold Graph Neural Networks](https://mlsanity.com/paper/2512.14023/) (2025)
- [Manifold-Matching Autoencoders](https://mlsanity.com/paper/2603.16568/) (2026)
- [The Rashomon Effect for Visualizing High-Dimensional Data](https://mlsanity.com/paper/2604.00485/) (2026)
- [Sharp Reconstruction Bounds for Autoencoders Using the Same Forward Map](https://mlsanity.com/paper/2609.20333/) (2026)
- [Federated Deep Clustering Networks for High-Dimensional and Heterogeneous Data](https://mlsanity.com/paper/2609.21829/) (2026)

## Abstract (arXiv)

Visualization is a crucial step in exploratory data analysis. One possible approach is to train an autoencoder with low-dimensional latent space. Large network depth and width can help unfolding the data. However, such expressive networks can achieve low reconstruction error even when the latent representation is distorted. To avoid such misleading visualizations, we propose first a differential geometric perspective on the decoder, leading to insightful diagnostics for an embedding's distortion, and second a new regularizer mitigating such distortion. Our ``Geometric Autoencoder'' avoids stretching the embedding spuriously, so that the visualization captures the data structure more faithfully. It also flags areas where little distortion could not be achieved, thus guarding against misinterpretation.