# Unsupervised Anomaly Localization with Structural Feature-Autoencoders

arXiv: https://arxiv.org/abs/2208.10992 · HTML: https://mlsanity.com/paper/2208.10992/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 24 references that are in the map.
- First posted on arXiv 23 Aug 2022.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Unsupervised Anomaly Localization using Variational Auto-Encoders](https://mlsanity.com/paper/1907.02796/) (2019)
- [Detecting Outliers with Foreign Patch Interpolation](https://mlsanity.com/paper/2011.04197/) (2020)
- [Detecting Outliers with Poisson Image Interpolation](https://mlsanity.com/paper/2107.02622/) (2021)
- [Challenging Current Semi-Supervised Anomaly Segmentation Methods for Brain MRI](https://mlsanity.com/paper/2109.06023/) (2021)
- [Simple statistical methods for unsupervised brain anomaly detection on MRI are competitive to deep learning methods](https://mlsanity.com/paper/2011.12735/) (2020)

## Built on by (most cited first, 23 of 23)

- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Unsupervised Pathology Detection: A Deep Dive Into the State of the Art](https://mlsanity.com/paper/2303.00609/) (2023)
- [Ensembled Cold-Diffusion Restorations for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2407.06635/) (2024)
- [Leveraging the Mahalanobis Distance to enhance Unsupervised Brain MRI Anomaly Detection](https://mlsanity.com/paper/2407.12474/) (2024)
- [DISYRE: Diffusion-Inspired SYnthetic REstoration for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2311.15453/) (2023)
- [Diffusion Models with Ensembled Structure-Based Anomaly Scoring for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2403.14262/) (2024)
- [(Predictable) Performance Bias in Unsupervised Anomaly Detection](https://mlsanity.com/paper/2309.14198/) (2023)
- [Enhancing Reconstruction-Based Out-of-Distribution Detection in Brain MRI with Model and Metric Ensembles](https://mlsanity.com/paper/2412.17586/) (2024)
- [A Survey on Video Analytics in Cloud-Edge-Terminal Collaborative Systems](https://mlsanity.com/paper/2502.06581/) (2025)
- [Q-Former Autoencoder: A Modern Framework for Medical Anomaly Detection](https://mlsanity.com/paper/2507.18481/) (2025)
- [Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis](https://mlsanity.com/paper/2512.01534/) (2025)
- [A Principled Approach to Unsupervised Anomaly Detection](https://mlsanity.com/paper/2609.21800/) (2026)
- [Kernel PCA for Out-of-Distribution Detection: Non-Linear Kernel Selection and Approximation](https://mlsanity.com/paper/2505.15284/) (2025)
- [Quantitative Benchmarking of Anomaly Detection Methods in Digital Pathology](https://mlsanity.com/paper/2506.19234/) (2025)
- [Patch2Loc: Learning to Localize Patches for Unsupervised Brain Lesion Detection](https://mlsanity.com/paper/2506.22504/) (2025)
- [Unsupervised Deep Generative Models for Anomaly Detection in Neuroimaging: A Systematic Scoping Review](https://mlsanity.com/paper/2510.14462/) (2025)
- [How Low Can You Go? Surfacing Prototypical In-Distribution Samples for Unsupervised Anomaly Detection](https://mlsanity.com/paper/2312.03804/) (2023)
- [Rethinking Medical Anomaly Detection in Brain MRI: An Image Quality Assessment Perspective](https://mlsanity.com/paper/2408.08228/) (2024)
- [PathoSCOPE: Few-Shot Pathology Detection via Self-Supervised Contrastive Learning and Pathology-Informed Synthetic Embeddings](https://mlsanity.com/paper/2505.17614/) (2025)
- [Latent Anomaly Detection: Masked VQ-GAN for Unsupervised Segmentation in Medical CBCT](https://mlsanity.com/paper/2506.14209/) (2025)
- [Semantic Iterative Reconstruction: One-Shot Universal Anomaly Detection](https://mlsanity.com/paper/2603.23766/) (2026)
- [AUCp: Pseudo-AUC for Inference Model Selection with Unlabeled Validation Data in Abnormality Detection](https://mlsanity.com/paper/2606.08742/) (2026)
- [Quasi-Binarized Autoencoders: An Architecture-Independent Information Bottleneck for Medical Image Anomaly Detection](https://mlsanity.com/paper/2610.09670/) (2026)

## Abstract (arXiv)

Unsupervised Anomaly Detection has become a popular method to detect pathologies in medical images as it does not require supervision or labels for training. Most commonly, the anomaly detection model generates a "normal" version of an input image, and the pixel-wise lp-difference of the two is used to localize anomalies. However, large residuals often occur due to imperfect reconstruction of the complex anatomical structures present in most medical images. This method also fails to detect anomalies that are not characterized by large intensity differences to the surrounding tissue. We propose to tackle this problem using a feature-mapping function that transforms the input intensity images into a space with multiple channels where anomalies can be detected along different discriminative feature maps extracted from the original image. We then train an Autoencoder model in this space using structural similarity loss that does not only consider differences in intensity but also in contrast and structure. Our method significantly increases performance on two medical data sets for brain MRI. Code and experiments are available at https://github.com/FeliMe/feature-autoencoder