# On a Mechanism Framework of Autoencoders

arXiv: https://arxiv.org/abs/2208.06995 · HTML: https://mlsanity.com/paper/2208.06995/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 46 references that are in the map.
- First posted on arXiv 15 Aug 2022.

## Builds on (most cited first, 5 of 5)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [What's Hidden in a Randomly Weighted Neural Network?](https://mlsanity.com/paper/1911.13299/) (2019)
- [Dynamical Variational Autoencoders: A Comprehensive Review](https://mlsanity.com/paper/2008.12595/) (2020)
- [Theoretical Exploration of Solutions of Feedforward ReLU Networks](https://mlsanity.com/paper/2202.01919/) (2022)

## Built on by (most cited first, 3 of 3)

- [Universal Solutions of Feedforward ReLU Networks for Interpolations](https://mlsanity.com/paper/2208.07498/) (2022)
- [On an Interpretation of ResNets via Solution Constructions](https://mlsanity.com/paper/2212.05663/) (2022)
- [KEEC: Koopman Embedded Equivariant Control](https://mlsanity.com/paper/2312.01544/) (2023)

## Abstract (arXiv)

This paper proposes a theoretical framework on the mechanism of autoencoders. To the encoder part, under the main use of dimensionality reduction, we investigate its two fundamental properties: bijective maps and data disentangling. The general construction methods of an encoder that satisfies either or both of the above two properties are given. The generalization mechanism of autoencoders is modeled. Based on the theoretical framework above, we explain some experimental results of variational autoencoders, denoising autoencoders, and linear-unit autoencoders, with emphasis on the interpretation of the lower-dimensional representation of data via encoders; and the mechanism of image restoration through autoencoders is natural to be understood by those explanations. Compared to PCA and decision trees, the advantages of (generalized) autoencoders on dimensionality reduction and classification are demonstrated, respectively. Convolutional neural networks and randomly weighted neural networks are also interpreted by this framework.