# Deep Symmetric Autoencoders from the Eckart-Young-Schmidt Perspective

arXiv: https://arxiv.org/abs/2506.11641 · HTML: https://mlsanity.com/paper/2506.11641/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 40 references that are in the map.
- First posted on arXiv 13 Jun 2025.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [A comprehensive deep learning-based approach to reduced order modeling of nonlinear time-dependent parametrized PDEs](https://mlsanity.com/paper/2001.04001/) (2020)
- [Convergence Rates for Learning Linear Operators from Noisy Data](https://mlsanity.com/paper/2108.12515/) (2021)
- [The gap between theory and practice in function approximation with deep neural networks](https://mlsanity.com/paper/2001.07523/) (2020)
- [A graph convolutional autoencoder approach to model order reduction for parametrized PDEs](https://mlsanity.com/paper/2305.08573/) (2023)
- [Deep Learning for Reduced Order Modelling and Efficient Temporal Evolution of Fluid Simulations](https://mlsanity.com/paper/2107.04556/) (2021)
- [A Deep Learning approach to Reduced Order Modelling of Parameter Dependent Partial Differential Equations](https://mlsanity.com/paper/2103.06183/) (2021)
- [Learning Nonlinear Projections for Reduced-Order Modeling of Dynamical Systems using Constrained Autoencoders](https://mlsanity.com/paper/2307.15288/) (2023)
- [Benefits of Jointly Training Autoencoders: An Improved Neural Tangent Kernel Analysis](https://mlsanity.com/paper/1911.11983/) (2019)
- [Invertible Autoencoder for domain adaptation](https://mlsanity.com/paper/1802.06869/) (2018)
- [PCA-Initialized Deep Neural Networks Applied To Document Image Analysis](https://mlsanity.com/paper/1702.00177/) (2017)
- [Establishing strong imputation performance of a denoising autoencoder in a wide range of missing data problems](https://mlsanity.com/paper/2004.02584/) (2020)

## Built on by (most cited first, 2 of 2)

- [Deep Invertible Autoencoders for Dimensionality Reduction of Dynamical Systems](https://mlsanity.com/paper/2603.13496/) (2026)
- [Convolutional Symmetric AutoEncoders: enhancing latent stability via differential geometry](https://mlsanity.com/paper/2607.00669/) (2026)

## Abstract (arXiv)

Deep autoencoders have become a fundamental tool in various machine learning applications, ranging from dimensionality reduction and reduced order modeling of partial differential equations to anomaly detection and neural machine translation. Despite their empirical success, a solid theoretical foundation for their expressiveness remains elusive, particularly when compared to classical projection-based techniques. In this work, we aim to take a step forward in this direction by presenting a comprehensive analysis of what we refer to as symmetric autoencoders, a broad class of deep learning architectures that have surfaced frequently in the recent literature. Specifically, we introduce a formal distinction between different classes of symmetric architectures, analyzing their strengths and limitations from a mathematical perspective. For instance, we show that the reconstruction error of symmetric autoencoders with orthonormality constraints can be understood by leveraging the well-renowned Eckart-Young-Schmidt (EYS) theorem. As a byproduct of our analysis, we end up developing the EYS initialization strategy for symmetric autoencoders, which is based on an iterated application of the Singular Value Decomposition (SVD). To validate our findings, we conduct a series of numerical experiments where we benchmark our proposal against conventional deep autoencoders, discussing the importance of model design and initialization.