# Empirical comparison between autoencoders and traditional dimensionality reduction methods

arXiv: https://arxiv.org/abs/2103.04874 · HTML: https://mlsanity.com/paper/2103.04874/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 8 Mar 2021.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)

## Built on by (most cited first, 13 of 13)

- [Explainable AI for Bioinformatics: Methods, Tools, and Applications](https://mlsanity.com/paper/2212.13261/) (2022)
- [Doubly Robust Conditional Independence Testing with Generative Neural Networks](https://mlsanity.com/paper/2407.17694/) (2024)
- [Efficient Data Compression for 3D Sparse TPC via Bicephalous Convolutional Autoencoder](https://mlsanity.com/paper/2111.05423/) (2021)
- [Interpreting Black-box Machine Learning Models for High Dimensional Datasets](https://mlsanity.com/paper/2208.13405/) (2022)
- [Machine Learning Methods for Small Data and Upstream Bioprocessing Applications: A Comprehensive Review](https://mlsanity.com/paper/2506.12322/) (2025)
- [Monitoring MBE substrate deoxidation via RHEED image-sequence analysis by deep learning](https://mlsanity.com/paper/2210.03430/) (2022)
- [Neural Network Characterization and Entropy Regulated Data Balancing through Principal Component Analysis](https://mlsanity.com/paper/2312.01392/) (2023)
- [DeepProphet2 -- A Deep Learning Gene Recommendation Engine](https://mlsanity.com/paper/2208.01918/) (2022)
- [Using linear initialisation to improve speed of convergence and fully-trained error in Autoencoders](https://mlsanity.com/paper/2311.10699/) (2023)
- [Out-of-Core Dimensionality Reduction for Large Data via Out-of-Sample Extensions](https://mlsanity.com/paper/2408.04129/) (2024)
- [Enhancing Obsolescence Forecasting with Deep Generative Data Augmentation: A Semi-Supervised Framework for Low-Data Industrial Applications](https://mlsanity.com/paper/2505.01261/) (2025)
- [Polyra Swarms: A Shape-Based Approach to Machine Learning](https://mlsanity.com/paper/2506.13217/) (2025)
- [Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality](https://mlsanity.com/paper/2509.17543/) (2025)

## Abstract (arXiv)

In order to process efficiently ever-higher dimensional data such as images, sentences, or audio recordings, one needs to find a proper way to reduce the dimensionality of such data. In this regard, SVD-based methods including PCA and Isomap have been extensively used. Recently, a neural network alternative called autoencoder has been proposed and is often preferred for its higher flexibility. This work aims to show that PCA is still a relevant technique for dimensionality reduction in the context of classification. To this purpose, we evaluated the performance of PCA compared to Isomap, a deep autoencoder, and a variational autoencoder. Experiments were conducted on three commonly used image datasets: MNIST, Fashion-MNIST, and CIFAR-10. The four different dimensionality reduction techniques were separately employed on each dataset to project data into a low-dimensional space. Then a k-NN classifier was trained on each projection with a cross-validated random search over the number of neighbours. Interestingly, our experiments revealed that k-NN achieved comparable accuracy on PCA and both autoencoders' projections provided a big enough dimension. However, PCA computation time was two orders of magnitude faster than its neural network counterparts.