# MSS-PAE: Saving Autoencoder-based Outlier Detection from Unexpected Reconstruction

arXiv: https://arxiv.org/abs/2304.00709 · HTML: https://mlsanity.com/paper/2304.00709/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 87 references that are in the map.
- First posted on arXiv 3 Apr 2023.

## Builds on (most cited first, 24 of 24)

- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [Do Deep Generative Models Know What They Don't Know?](https://mlsanity.com/paper/1810.09136/) (2018)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Deep Semi-Supervised Anomaly Detection](https://mlsanity.com/paper/1906.02694/) (2019)
- [PyOD: A Python Toolbox for Scalable Outlier Detection](https://mlsanity.com/paper/1901.01588/) (2019)
- [ECOD: Unsupervised Outlier Detection Using Empirical Cumulative Distribution Functions](https://mlsanity.com/paper/2201.00382/) (2022)
- [Uncertainty in Neural Networks: Approximately Bayesian Ensembling](https://mlsanity.com/paper/1810.05546/) (2018)
- [Deep Isolation Forest for Anomaly Detection](https://mlsanity.com/paper/2206.06602/) (2022)
- [Stochastic gradient Markov chain Monte Carlo](https://mlsanity.com/paper/1907.06986/) (2019)
- [Anomaly Detection With Conditional Variational Autoencoders](https://mlsanity.com/paper/2010.05531/) (2020)
- [Uncertainty-Aware Deep Classifiers using Generative Models](https://mlsanity.com/paper/2006.04183/) (2020)
- [Bayesian Variational Autoencoders for Unsupervised Out-of-Distribution Detection](https://mlsanity.com/paper/1912.05651/) (2019)
- [Robust Subspace Recovery Layer for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.00152/) (2019)
- [Resisting Adversarial Attacks using Gaussian Mixture Variational Autoencoders](https://mlsanity.com/paper/1806.00081/) (2018)
- [Memory Augmented Generative Adversarial Networks for Anomaly Detection](https://mlsanity.com/paper/2002.02669/) (2020)
- [Interpreting Rate-Distortion of Variational Autoencoder and Using Model Uncertainty for Anomaly Detection](https://mlsanity.com/paper/2005.01889/) (2020)
- [Bayesian Autoencoders: Analysing and Fixing the Bernoulli likelihood for Out-of-Distribution Detection](https://mlsanity.com/paper/2107.13304/) (2021)
- [Bayesian autoencoders with uncertainty quantification: Towards trustworthy anomaly detection](https://mlsanity.com/paper/2202.12653/) (2022)

## Abstract (arXiv)

AutoEncoders (AEs) are commonly used for machine learning tasks due to their intrinsic learning ability. This unique characteristic can be capitalized for Outlier Detection (OD). However conventional AE-based methods face the issue of overconfident decisions and unexpected reconstruction results of outliers, limiting their performance in OD. To mitigate these issues, the Mean Squared Error (MSE) and Negative Logarithmic Likelihood (NLL) were firstly analyzed, and the importance of incorporating aleatoric uncertainty to AE-based OD was elucidated. Then the Weighted Negative Logarithmic Likelihood (WNLL) was proposed to adjust for the effect of uncertainty for different OD scenarios. Moreover, the Mean-Shift Scoring (MSS) method was proposed to utilize the local relationship of data to reduce the issue of false inliers caused by AE. Experiments on 32 real-world OD datasets proved the effectiveness of the proposed methods. The combination of WNLL and MSS achieved 41% relative performance improvement compared to the best baseline. In addition, MSS improved the detection performance of multiple AE-based outlier detectors by an average of 20%. The proposed methods have the potential to advance AE's development in OD.