# Geometric instability of out of distribution data across autoencoder architecture

arXiv: https://arxiv.org/abs/2201.11902 · HTML: https://mlsanity.com/paper/2201.11902/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 17 of 17)

- [A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks](https://mlsanity.com/paper/1610.02136/) (2016)
- [Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks](https://mlsanity.com/paper/1706.02690/) (2017)
- [Out-of-Distribution Generalization via Risk Extrapolation (REx)](https://mlsanity.com/paper/2003.00688/) (2020)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [Latent Space Oddity: on the Curvature of Deep Generative Models](https://mlsanity.com/paper/1710.11379/) (2017)
- [Likelihood Regret: An Out-of-Distribution Detection Score For Variational Auto-encoder](https://mlsanity.com/paper/2003.02977/) (2020)
- [Predictive Inequity in Object Detection](https://mlsanity.com/paper/1902.11097/) (2019)
- [Improving Reconstruction Autoencoder Out-of-distribution Detection with Mahalanobis Distance](https://mlsanity.com/paper/1812.02765/) (2018)
- [Hierarchical VAEs Know What They Don't Know](https://mlsanity.com/paper/2102.08248/) (2021)
- [Geometrically Enriched Latent Spaces](https://mlsanity.com/paper/2008.00565/) (2020)
- [Metrics for Probabilistic Geometries](https://mlsanity.com/paper/1411.7432/) (2014)
- [Autoencoding Under Normalization Constraints](https://mlsanity.com/paper/2105.05735/) (2021)
- [Only Bayes should learn a manifold (on the estimation of differential geometric structure from data)](https://mlsanity.com/paper/1806.04994/) (2018)
- [Expected path length on random manifolds](https://mlsanity.com/paper/1908.07377/) (2019)
- [Memorization in Overparameterized Autoencoders](https://mlsanity.com/paper/1810.10333/) (2018)
- [Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalize](https://mlsanity.com/paper/2107.06386/) (2021)

## Built on by (most cited first, 1 of 1)

- [Stable CDE Autoencoders with Acuity Regularization for Offline Reinforcement Learning in Sepsis Treatment](https://mlsanity.com/paper/2506.15019/) (2025)

## Abstract (arXiv)

We study the map learned by a family of autoencoders trained on MNIST, and evaluated on ten different data sets created by the random selection of pixel values according to ten different distributions. Specifically, we study the eigenvalues of the Jacobians defined by the weight matrices of the autoencoder at each training and evaluation point. For high enough latent dimension, we find that each autoencoder reconstructs all the evaluation data sets as similar generalized characters, but that this reconstructed generalized character changes across autoencoder. Eigenvalue analysis shows that even when the reconstructed image appears to be an MNIST character for all out of distribution data sets, not all have latent representations that are close to the latent representation of MNIST characters. All told, the eigenvalue analysis demonstrated a great deal of geometric instability of the autoencoder both as a function on out of distribution inputs, and across architectures on the same set of inputs.