# IterMask2: Iterative Unsupervised Anomaly Segmentation via Spatial and Frequency Masking for Brain Lesions in MRI

arXiv: https://arxiv.org/abs/2406.02422 · HTML: https://mlsanity.com/paper/2406.02422/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 22 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 11 of 11)

- [The RSNA-ASNR-MICCAI BraTS 2021 Benchmark on Brain Tumor Segmentation and Radiogenomic Classification](https://mlsanity.com/paper/2107.02314/) (2021)
- [Efficient Multi-Scale 3D CNN with Fully Connected CRF for Accurate Brain Lesion Segmentation](https://mlsanity.com/paper/1603.05959/) (2016)
- [Brain Tumor Segmentation with Deep Neural Networks](https://mlsanity.com/paper/1505.03540/) (2015)
- [nnU-Net for Brain Tumor Segmentation](https://mlsanity.com/paper/2011.00848/) (2020)
- [Fast Unsupervised Brain Anomaly Detection and Segmentation with Diffusion Models](https://mlsanity.com/paper/2206.03461/) (2022)
- [Unsupervised Anomaly Localization using Variational Auto-Encoders](https://mlsanity.com/paper/1907.02796/) (2019)
- [Masked Frequency Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2206.07706/) (2022)
- [Mask, Stitch, and Re-Sample: Enhancing Robustness and Generalizability in Anomaly Detection through Automatic Diffusion Models](https://mlsanity.com/paper/2305.19643/) (2023)
- [Unsupervised brain lesion segmentation from MRI using a convolutional autoencoder](https://mlsanity.com/paper/1811.09655/) (2018)
- [Parallel Multi-Dimensional LSTM, With Application to Fast Biomedical Volumetric Image Segmentation](https://mlsanity.com/paper/1506.07452/) (2015)
- [Modality Cycles with Masked Conditional Diffusion for Unsupervised Anomaly Segmentation in MRI](https://mlsanity.com/paper/2308.16150/) (2023)

## Built on by (most cited first, 9 of 9)

- [MAD-AD: Masked Diffusion for Unsupervised Brain Anomaly Detection](https://mlsanity.com/paper/2502.16943/) (2025)
- [REFLECT: Rectified Flows for Efficient Brain Anomaly Correction Transport](https://mlsanity.com/paper/2508.02889/) (2025)
- [Constitution or Collapse? Exploring Constitutional AI with Llama 3-8B](https://mlsanity.com/paper/2504.04918/) (2025)
- [Deep Unsupervised Anomaly Detection in Brain Imaging: Large-Scale Benchmarking and Bias Analysis](https://mlsanity.com/paper/2512.01534/) (2025)
- [ART-ASyn: Anatomy-aware Realistic Texture-based Anomaly Synthesis Framework for Chest X-Rays](https://mlsanity.com/paper/2512.00310/) (2025)
- [Bilateral Hippocampi Segmentation in Low Field MRIs Using Mutual Feature Learning via Dual-Views](https://mlsanity.com/paper/2410.17502/) (2024)
- [U2AD: Uncertainty-based Unsupervised Anomaly Detection Framework for Detecting T2 Hyperintensity in MRI Spinal Cord](https://mlsanity.com/paper/2503.13400/) (2025)
- [AREPAS: Anomaly Detection in Fine-Grained Anatomy with Reconstruction-Based Semantic Patch-Scoring](https://mlsanity.com/paper/2509.12905/) (2025)
- [Unsupervised Brain Anomaly Detection as a Bayesian Inverse Problem with Diffusion Prior](https://mlsanity.com/paper/2609.24265/) (2026)

## Abstract (arXiv)

Unsupervised anomaly segmentation approaches to pathology segmentation train a model on images of healthy subjects, that they define as the 'normal' data distribution. At inference, they aim to segment any pathologies in new images as 'anomalies', as they exhibit patterns that deviate from those in 'normal' training data. Prevailing methods follow the 'corrupt-and-reconstruct' paradigm. They intentionally corrupt an input image, reconstruct it to follow the learned 'normal' distribution, and subsequently segment anomalies based on reconstruction error. Corrupting an input image, however, inevitably leads to suboptimal reconstruction even of normal regions, causing false positives. To alleviate this, we propose a novel iterative spatial mask-refining strategy IterMask2. We iteratively mask areas of the image, reconstruct them, and update the mask based on reconstruction error. This iterative process progressively adds information about areas that are confidently normal as per the model. The increasing content guides reconstruction of nearby masked areas, improving reconstruction of normal tissue under these areas, reducing false positives. We also use high-frequency image content as an auxiliary input to provide additional structural information for masked areas. This further improves reconstruction error of normal in comparison to anomalous areas, facilitating segmentation of the latter. We conduct experiments on several brain lesion datasets and demonstrate effectiveness of our method. Code is available at: https://github.com/ZiyunLiang/IterMask2