# Looking at the whole picture: constrained unsupervised anomaly segmentation

arXiv: https://arxiv.org/abs/2109.00482 · HTML: https://mlsanity.com/paper/2109.00482/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 40 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Unsupervised Anomaly Detection with Generative Adversarial Networks to Guide Marker Discovery](https://mlsanity.com/paper/1703.05921/) (2017)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Improving Unsupervised Defect Segmentation by Applying Structural Similarity to Autoencoders](https://mlsanity.com/paper/1807.02011/) (2018)
- [Tell Me Where to Look: Guided Attention Inference Network](https://mlsanity.com/paper/1802.10171/) (2018)
- [Latent Space Autoregression for Novelty Detection](https://mlsanity.com/paper/1807.01653/) (2018)
- [Autoencoders for Unsupervised Anomaly Segmentation in Brain MR Images: A Comparative Study](https://mlsanity.com/paper/2004.03271/) (2020)
- [Deep Autoencoding Models for Unsupervised Anomaly Segmentation in Brain MR Images](https://mlsanity.com/paper/1804.04488/) (2018)
- [Towards Visually Explaining Variational Autoencoders](https://mlsanity.com/paper/1911.07389/) (2019)
- [Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes](https://mlsanity.com/paper/1707.09465/) (2017)
- [Constrained Convolutional Neural Networks for Weakly Supervised Segmentation](https://mlsanity.com/paper/1506.03648/) (2015)
- [Prior-aware Neural Network for Partially-Supervised Multi-Organ Segmentation](https://mlsanity.com/paper/1904.06346/) (2019)
- [Unsupervised Detection of Lesions in Brain MRI using constrained adversarial auto-encoders](https://mlsanity.com/paper/1806.04972/) (2018)
- [Unsupervised Lesion Detection via Image Restoration with a Normative Prior](https://mlsanity.com/paper/2005.00031/) (2020)
- [Context-encoding Variational Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1812.05941/) (2018)
- [Constrained-CNN losses for weakly supervised segmentation](https://mlsanity.com/paper/1805.04628/) (2018)
- [Constrained Deep Weak Supervision for Histopathology Image Segmentation](https://mlsanity.com/paper/1701.00794/) (2017)
- [Iterative energy-based projection on a normal data manifold for anomaly localization](https://mlsanity.com/paper/2002.03734/) (2020)
- [An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection](https://mlsanity.com/paper/1810.10850/) (2018)
- [Constrained Deep Networks: Lagrangian Optimization via Log-Barrier Extensions](https://mlsanity.com/paper/1904.04205/) (2019)
- [Training Adversarial Discriminators for Cross-channel Abnormal Event Detection in Crowds](https://mlsanity.com/paper/1706.07680/) (2017)
- [Curriculum semi-supervised segmentation](https://mlsanity.com/paper/1904.05236/) (2019)
- [Learning to Play in a Day: Faster Deep Reinforcement Learning by Optimality Tightening](https://mlsanity.com/paper/1611.01606/) (2016)
- [AVID: Adversarial Visual Irregularity Detection](https://mlsanity.com/paper/1805.09521/) (2018)
- [Discretely-constrained deep network for weakly supervised segmentation](https://mlsanity.com/paper/1908.05770/) (2019)
- [Constrained domain adaptation for Image segmentation](https://mlsanity.com/paper/1908.02996/) (2019)
- [Pathology Segmentation using Distributional Differences to Images of Healthy Origin](https://mlsanity.com/paper/1805.10344/) (2018)

## Built on by (most cited first, 4 of 4)

- [Constrained unsupervised anomaly segmentation](https://mlsanity.com/paper/2203.01671/) (2022)
- [An efficient semi-supervised quality control system trained using physics-based MRI-artefact generators and adversarial training](https://mlsanity.com/paper/2206.03359/) (2022)
- [Exploring Different Levels of Supervision for Detecting and Localizing Solar Panels on Remote Sensing Imagery](https://mlsanity.com/paper/2309.10421/) (2023)
- [Attention to detail: inter-resolution knowledge distillation](https://mlsanity.com/paper/2401.06010/) (2024)

## Abstract (arXiv)

Current unsupervised anomaly localization approaches rely on generative models to learn the distribution of normal images, which is later used to identify potential anomalous regions derived from errors on the reconstructed images. However, a main limitation of nearly all prior literature is the need of employing anomalous images to set a class-specific threshold to locate the anomalies. This limits their usability in realistic scenarios, where only normal data is typically accessible. Despite this major drawback, only a handful of works have addressed this limitation, by integrating supervision on attention maps during training. In this work, we propose a novel formulation that does not require accessing images with abnormalities to define the threshold. Furthermore, and in contrast to very recent work, the proposed constraint is formulated in a more principled manner, leveraging well-known knowledge in constrained optimization. In particular, the equality constraint on the attention maps in prior work is replaced by an inequality constraint, which allows more flexibility. In addition, to address the limitations of penalty-based functions we employ an extension of the popular log-barrier methods to handle the constraint. Comprehensive experiments on the popular BRATS'19 dataset demonstrate that the proposed approach substantially outperforms relevant literature, establishing new state-of-the-art results for unsupervised lesion segmentation.