# AMAE: Adaptation of Pre-Trained Masked Autoencoder for Dual-Distribution Anomaly Detection in Chest X-Rays

arXiv: https://arxiv.org/abs/2307.12721 · HTML: https://mlsanity.com/paper/2307.12721/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [PaDiM: a Patch Distribution Modeling Framework for Anomaly Detection and Localization](https://mlsanity.com/paper/2011.08785/) (2020)
- [Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection](https://mlsanity.com/paper/1904.02639/) (2019)
- [CutPaste: Self-Supervised Learning for Anomaly Detection and Localization](https://mlsanity.com/paper/2104.04015/) (2021)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [VinDr-CXR: An open dataset of chest X-rays with radiologist's annotations](https://mlsanity.com/paper/2012.15029/) (2020)
- [Natural Synthetic Anomalies for Self-Supervised Anomaly Detection and Localization](https://mlsanity.com/paper/2109.15222/) (2021)
- [SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2012.11460/) (2020)
- [Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning](https://mlsanity.com/paper/2102.10680/) (2021)
- [Deep One-Class Classification via Interpolated Gaussian Descriptor](https://mlsanity.com/paper/2101.10043/) (2021)
- [Delving into Masked Autoencoders for Multi-Label Thorax Disease Classification](https://mlsanity.com/paper/2210.12843/) (2022)
- [Constrained Contrastive Distribution Learning for Unsupervised Anomaly Detection and Localisation in Medical Images](https://mlsanity.com/paper/2103.03423/) (2021)
- [Detecting Outliers with Foreign Patch Interpolation](https://mlsanity.com/paper/2011.04197/) (2020)
- [Detecting Outliers with Poisson Image Interpolation](https://mlsanity.com/paper/2107.02622/) (2021)
- [Dual-Distribution Discrepancy for Anomaly Detection in Chest X-Rays](https://mlsanity.com/paper/2206.03935/) (2022)
- [Anatomy-aware Self-supervised Learning for Anomaly Detection in Chest Radiographs](https://mlsanity.com/paper/2205.04282/) (2022)
- [Self-Taught Semi-Supervised Anomaly Detection on Upper Limb X-rays](https://mlsanity.com/paper/2102.09895/) (2021)

## Built on by (most cited first, 8 of 8)

- [Masked Image Modeling: A Survey](https://mlsanity.com/paper/2408.06687/) (2024)
- [MedIAnomaly: A comparative study of anomaly detection in medical images](https://mlsanity.com/paper/2404.04518/) (2024)
- [Revisiting Deep Ensemble Uncertainty for Enhanced Medical Anomaly Detection](https://mlsanity.com/paper/2409.17485/) (2024)
- [Position-Guided Prompt Learning for Anomaly Detection in Chest X-Rays](https://mlsanity.com/paper/2405.11976/) (2024)
- [Weakly Supervised Anomaly Detection for Chest X-Ray Image](https://mlsanity.com/paper/2311.09642/) (2023)
- [Spatial-aware Attention Generative Adversarial Network for Semi-supervised Anomaly Detection in Medical Image](https://mlsanity.com/paper/2405.12872/) (2024)
- [Unsupervised Hybrid framework for ANomaly Detection (HAND) -- applied to Screening Mammogram](https://mlsanity.com/paper/2409.11534/) (2024)
- [MTL-MAD: Multi-Task Learners are Effective Medical Anomaly Detectors](https://mlsanity.com/paper/2605.05891/) (2026)

## Abstract (arXiv)

Unsupervised anomaly detection in medical images such as chest radiographs is stepping into the spotlight as it mitigates the scarcity of the labor-intensive and costly expert annotation of anomaly data. However, nearly all existing methods are formulated as a one-class classification trained only on representations from the normal class and discard a potentially significant portion of the unlabeled data. This paper focuses on a more practical setting, dual distribution anomaly detection for chest X-rays, using the entire training data, including both normal and unlabeled images. Inspired by a modern self-supervised vision transformer model trained using partial image inputs to reconstruct missing image regions -- we propose AMAE, a two-stage algorithm for adaptation of the pre-trained masked autoencoder (MAE). Starting from MAE initialization, AMAE first creates synthetic anomalies from only normal training images and trains a lightweight classifier on frozen transformer features. Subsequently, we propose an adaptation strategy to leverage unlabeled images containing anomalies. The adaptation scheme is accomplished by assigning pseudo-labels to unlabeled images and using two separate MAE based modules to model the normative and anomalous distributions of pseudo-labeled images. The effectiveness of the proposed adaptation strategy is evaluated with different anomaly ratios in an unlabeled training set. AMAE leads to consistent performance gains over competing self-supervised and dual distribution anomaly detection methods, setting the new state-of-the-art on three public chest X-ray benchmarks: RSNA, NIH-CXR, and VinDr-CXR.