# Unsupervised Anomaly Instance Segmentation for Baggage Threat Recognition

arXiv: https://arxiv.org/abs/2107.07333 · HTML: https://mlsanity.com/paper/2107.07333/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 77 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Focal Loss for Dense Object Detection](https://mlsanity.com/paper/1708.02002/) (2017)
- [FDA: Fourier Domain Adaptation for Semantic Segmentation](https://mlsanity.com/paper/2004.05498/) (2020)
- [GANomaly: Semi-Supervised Anomaly Detection via Adversarial Training](https://mlsanity.com/paper/1805.06725/) (2018)
- [Meta-Transfer Learning for Few-Shot Learning](https://mlsanity.com/paper/1812.02391/) (2018)
- [Skip-GANomaly: Skip Connected and Adversarially Trained Encoder-Decoder Anomaly Detection](https://mlsanity.com/paper/1901.08954/) (2019)
- [SIXray : A Large-scale Security Inspection X-ray Benchmark for Prohibited Item Discovery in Overlapping Images](https://mlsanity.com/paper/1901.00303/) (2019)
- [Occluded Prohibited Items Detection: an X-ray Security Inspection Benchmark and De-occlusion Attention Module](https://mlsanity.com/paper/2004.08656/) (2020)
- [Towards Automatic Threat Detection: A Survey of Advances of Deep Learning within X-ray Security Imaging](https://mlsanity.com/paper/2001.01293/) (2020)
- [Clinically Verified Hybrid Deep Learning System for Retinal Ganglion Cells Aware Grading of Glaucomatous Progression](https://mlsanity.com/paper/2010.03872/) (2020)
- [Evaluation of a Dual Convolutional Neural Network Architecture for Object-wise Anomaly Detection in Cluttered X-ray Security Imagery](https://mlsanity.com/paper/1904.05304/) (2019)
- [Cascaded Structure Tensor Framework for Robust Identification of Heavily Occluded Baggage Items from X-ray Scans](https://mlsanity.com/paper/2004.06780/) (2020)
- [SIP-SegNet: A Deep Convolutional Encoder-Decoder Network for Joint Semantic Segmentation and Extraction of Sclera, Iris and Pupil based on Periocular Region Suppression](https://mlsanity.com/paper/2003.00825/) (2020)
- [Evaluating the Transferability and Adversarial Discrimination of Convolutional Neural Networks for Threat Object Detection and Classification within X-Ray Security Imagery](https://mlsanity.com/paper/1911.08966/) (2019)
- [Trainable Structure Tensors for Autonomous Baggage Threat Detection Under Extreme Occlusion](https://mlsanity.com/paper/2009.13158/) (2020)
- [A Dilated Residual Hierarchically Fashioned Segmentation Framework for Extracting Gleason Tissues and Grading Prostate Cancer from Whole Slide Images](https://mlsanity.com/paper/2011.00527/) (2020)
- [Multi-Class 3D Object Detection Within Volumetric 3D Computed Tomography Baggage Security Screening Imagery](https://mlsanity.com/paper/2008.01218/) (2020)
- [Over-sampling De-occlusion Attention Network for Prohibited Items Detection in Noisy X-ray Images](https://mlsanity.com/paper/2103.00809/) (2021)
- [Detection of concealed cars in complex cargo X-ray imagery using Deep Learning](https://mlsanity.com/paper/1606.08078/) (2016)
- [Contraband Materials Detection Within Volumetric 3D Computed Tomography Baggage Security Screening Imagery](https://mlsanity.com/paper/2012.11753/) (2020)
- [A Reference Architecture for Plausible Threat Image Projection (TIP) Within 3D X-ray Computed Tomography Volumes](https://mlsanity.com/paper/2001.05459/) (2020)
- [On the Evaluation of Prohibited Item Classification and Detection in Volumetric 3D Computed Tomography Baggage Security Screening Imagery](https://mlsanity.com/paper/2003.12625/) (2020)

## Built on by (most cited first, 8 of 8)

- [Vision-Based Autonomous Navigation for Unmanned Surface Vessel in Extreme Marine Conditions](https://mlsanity.com/paper/2308.04283/) (2023)
- [Tensor Pooling Driven Instance Segmentation Framework for Baggage Threat Recognition](https://mlsanity.com/paper/2108.09603/) (2021)
- [STING-BEE: Towards Vision-Language Model for Real-World X-ray Baggage Security Inspection](https://mlsanity.com/paper/2504.02823/) (2025)
- [An Incremental Learning Approach to Automatically Recognize Pulmonary Diseases from the Multi-vendor Chest Radiographs](https://mlsanity.com/paper/2201.02574/) (2022)
- [Incremental Cross-Domain Adaptation for Robust Retinopathy Screening via Bayesian Deep Learning](https://mlsanity.com/paper/2110.09319/) (2021)
- [A Novel Incremental Learning Driven Instance Segmentation Framework to Recognize Highly Cluttered Instances of the Contraband Items](https://mlsanity.com/paper/2201.02560/) (2022)
- [Automated segmentation and extraction of posterior eye segment using OCT scans](https://mlsanity.com/paper/2109.10000/) (2021)
- [Temporal Fusion Based Mutli-scale Semantic Segmentation for Detecting Concealed Baggage Threats](https://mlsanity.com/paper/2111.02651/) (2021)

## Abstract (arXiv)

Identifying potential threats concealed within the baggage is of prime concern for the security staff. Many researchers have developed frameworks that can detect baggage threats from X-ray scans. However, to the best of our knowledge, all of these frameworks require extensive training on large-scale and well-annotated datasets, which are hard to procure in the real world. This paper presents a novel unsupervised anomaly instance segmentation framework that recognizes baggage threats, in X-ray scans, as anomalies without requiring any ground truth labels. Furthermore, thanks to its stylization capacity, the framework is trained only once, and at the inference stage, it detects and extracts contraband items regardless of their scanner specifications. Our one-staged approach initially learns to reconstruct normal baggage content via an encoder-decoder network utilizing a proposed stylization loss function. The model subsequently identifies the abnormal regions by analyzing the disparities within the original and the reconstructed scans. The anomalous regions are then clustered and post-processed to fit a bounding box for their localization. In addition, an optional classifier can also be appended with the proposed framework to recognize the categories of these extracted anomalies. A thorough evaluation of the proposed system on four public baggage X-ray datasets, without any re-training, demonstrates that it achieves competitive performance as compared to the conventional fully supervised methods (i.e., the mean average precision score of 0.7941 on SIXray, 0.8591 on GDXray, 0.7483 on OPIXray, and 0.5439 on COMPASS-XP dataset) while outperforming state-of-the-art semi-supervised and unsupervised baggage threat detection frameworks by 67.37%, 32.32%, 47.19%, and 45.81% in terms of F1 score across SIXray, GDXray, OPIXray, and COMPASS-XP datasets, respectively.