# Detecting Anomalous Faces with 'No Peeking' Autoencoders

arXiv: https://arxiv.org/abs/1802.05798 · HTML: https://mlsanity.com/paper/1802.05798/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Feb 2018.

## Built on by (most cited first, 5 of 5)

- [Inpainting Transformer for Anomaly Detection](https://mlsanity.com/paper/2104.13897/) (2021)
- [Self-Supervised Training with Autoencoders for Visual Anomaly Detection](https://mlsanity.com/paper/2206.11723/) (2022)
- [Vision Augmentation Prediction Autoencoder with Attention Design (VAPAAD)](https://mlsanity.com/paper/2404.10096/) (2024)
- [Noise & pattern: identity-anchored Tikhonov regularization for robust structural anomaly detection](https://mlsanity.com/paper/2511.07233/) (2025)
- [Rethinking Structural Anomaly Detection: From Decision Boundaries to Projection Operators](https://mlsanity.com/paper/2606.15280/) (2026)

## Abstract (arXiv)

Detecting anomalous faces has important applications. For example, a system might tell when a train driver is incapacitated by a medical event, and assist in adopting a safe recovery strategy. These applications are demanding, because they require accurate detection of rare anomalies that may be seen only at runtime. Such a setting causes supervised methods to perform poorly. We describe a method for detecting an anomalous face image that meets these requirements. We construct a feature vector that reliably has large entries for anomalous images, then use various simple unsupervised methods to score the image based on the feature. Obvious constructions (autoencoder codes; autoencoder residuals) are defeated by a 'peeking' behavior in autoencoders. Our feature construction removes rectangular patches from the image, predicts the likely content of the patch conditioned on the rest of the image using a specially trained autoencoder, then compares the result to the image. High scores suggest that the patch was difficult for an autoencoder to predict, and so is likely anomalous. We demonstrate that our method can identify real anomalous face images in pools of typical images, taken from celeb-A, that is much larger than usual in state-of-the-art experiments. A control experiment based on our method with another set of normal celebrity images - a 'typical set', but nonceleb-A are not identified as anomalous; confirms this is not due to special properties of celeb-A.