# Probabilistic Robust Autoencoders for Outlier Detection

arXiv: https://arxiv.org/abs/2110.00494 · HTML: https://mlsanity.com/paper/2110.00494/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 65 references that are in the map.
- First posted on arXiv 1 Oct 2021.

## Builds on (most cited first, 21 of 21)

- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Anomaly Detection with Outlier Exposure](https://mlsanity.com/paper/1812.04606/) (2018)
- [Deep Learning and the Information Bottleneck Principle](https://mlsanity.com/paper/1503.02406/) (2015)
- [Using Self-Supervised Learning Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1906.12340/) (2019)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [Classification-Based Anomaly Detection for General Data](https://mlsanity.com/paper/2005.02359/) (2020)
- [PANDA: Adapting Pretrained Features for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2010.05903/) (2020)
- [The Convergence Rate of Neural Networks for Learned Functions of Different Frequencies](https://mlsanity.com/paper/1906.00425/) (2019)
- [DROCC: Deep Robust One-Class Classification](https://mlsanity.com/paper/2002.12718/) (2020)
- [Generative Adversarial Active Learning for Unsupervised Outlier Detection](https://mlsanity.com/paper/1809.10816/) (2018)
- [From Principal Subspaces to Principal Components with Linear Autoencoders](https://mlsanity.com/paper/1804.10253/) (2018)
- [One-Class Classification: A Survey](https://mlsanity.com/paper/2101.03064/) (2021)
- [LSCP: Locally Selective Combination in Parallel Outlier Ensembles](https://mlsanity.com/paper/1812.01528/) (2018)
- [Robust Subspace Recovery Layer for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.00152/) (2019)
- [An Overview of Robust Subspace Recovery](https://mlsanity.com/paper/1803.01013/) (2018)
- [Fast, Robust and Non-convex Subspace Recovery](https://mlsanity.com/paper/1406.6145/) (2014)
- [LOCA: LOcal Conformal Autoencoder for standardized data coordinates](https://mlsanity.com/paper/2004.07234/) (2020)
- [A Well-Tempered Landscape for Non-convex Robust Subspace Recovery](https://mlsanity.com/paper/1706.03896/) (2017)
- [Robust Variational Autoencoders for Outlier Detection and Repair of Mixed-Type Data](https://mlsanity.com/paper/1907.06671/) (2019)
- [Non-Parametric Estimation of Manifolds from Noisy Data](https://mlsanity.com/paper/2105.04754/) (2021)
- [PCA-Based Out-of-Sample Extension for Dimensionality Reduction](https://mlsanity.com/paper/1511.00831/) (2015)

## Built on by (most cited first, 7 of 7)

- [Unsupervised Anomaly Detection with Rejection](https://mlsanity.com/paper/2305.13189/) (2023)
- [Anomaly Detection with Variance Stabilized Density Estimation](https://mlsanity.com/paper/2306.00582/) (2023)
- [Contextual Feature Selection with Conditional Stochastic Gates](https://mlsanity.com/paper/2312.14254/) (2023)
- [TabADM: Unsupervised Tabular Anomaly Detection with Diffusion Models](https://mlsanity.com/paper/2307.12336/) (2023)
- [Unveiling Multiple Descents in Unsupervised Autoencoders](https://mlsanity.com/paper/2406.11703/) (2024)
- [Neuronal Cell Type Classification using Deep Learning](https://mlsanity.com/paper/2306.00528/) (2023)
- [AI-based particle track identification in scintillating fibres read out with imaging sensors](https://mlsanity.com/paper/2410.10519/) (2024)

## Abstract (arXiv)

Anomalies (or outliers) are prevalent in real-world empirical observations and potentially mask important underlying structures. Accurate identification of anomalous samples is crucial for the success of downstream data analysis tasks. To automatically identify anomalies, we propose Probabilistic Robust AutoEncoder (PRAE). PRAE aims to simultaneously remove outliers and identify a low-dimensional representation for the inlier samples. We first present the Robust AutoEncoder (RAE) objective as a minimization problem for splitting the data into inliers and outliers. Our objective is designed to exclude outliers while including a subset of samples (inliers) that can be effectively reconstructed using an AutoEncoder (AE). RAE minimizes the autoencoder's reconstruction error while incorporating as many samples as possible. This could be formulated via regularization by subtracting an ℓ0 norm counting the number of selected samples from the reconstruction term. Unfortunately, this leads to an intractable combinatorial problem. Therefore, we propose two probabilistic relaxations of RAE, which are differentiable and alleviate the need for a combinatorial search. We prove that the solution to the PRAE problem is equivalent to the solution of RAE. We use synthetic data to show that PRAE can accurately remove outliers in a wide range of contamination levels. Finally, we demonstrate that using PRAE for anomaly detection leads to state-of-the-art results on various benchmark datasets.