# You Only Train Once: Learning a General Anomaly Enhancement Network with Random Masks for Hyperspectral Anomaly Detection

arXiv: https://arxiv.org/abs/2303.18001 · HTML: https://mlsanity.com/paper/2303.18001/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 75 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [Improved Regularization of Convolutional Neural Networks with Cutout](https://mlsanity.com/paper/1708.04552/) (2017)
- [Swin-Unet: Unet-like Pure Transformer for Medical Image Segmentation](https://mlsanity.com/paper/2105.05537/) (2021)
- [Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet](https://mlsanity.com/paper/2101.11986/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [Plug-and-Play Image Restoration with Deep Denoiser Prior](https://mlsanity.com/paper/2008.13751/) (2020)
- [Gradient Magnitude Similarity Deviation: A Highly Efficient Perceptual Image Quality Index](https://mlsanity.com/paper/1308.3052/) (2013)
- [Dense Nested Attention Network for Infrared Small Target Detection](https://mlsanity.com/paper/2106.00487/) (2021)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)

## Built on by (most cited first, 12 of 12)

- [BSDM: Background Suppression Diffusion Model for Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2307.09861/) (2023)
- [SpecDETR: A transformer-based hyperspectral point object detection network](https://mlsanity.com/paper/2405.10148/) (2024)
- [Exploring Hyperspectral Anomaly Detection with Human Vision: A Small Target Aware Detector](https://mlsanity.com/paper/2401.01093/) (2024)
- [ERX: A Fast Real-Time Anomaly Detection Algorithm for Hyperspectral Line Scanning](https://mlsanity.com/paper/2408.14947/) (2024)
- [Self-Supervised Learning for Real-World Object Detection: a Survey](https://mlsanity.com/paper/2410.07442/) (2024)
- [ADNet: A Large-Scale and Extensible Multi-Domain Benchmark for Anomaly Detection Across 380 Real-World Categories](https://mlsanity.com/paper/2511.20169/) (2025)
- [SUIT: Spatial-Spectral Union-Intersection Interaction Network for Hyperspectral Object Tracking](https://mlsanity.com/paper/2508.07250/) (2025)
- [Overcoming the Identity Mapping Problem in Self-Supervised Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2504.04115/) (2025)
- [CL-BioGAN: Biologically-Inspired Cross-Domain Continual Learning for Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2505.11796/) (2025)
- [Decoupled Complementary Spectral-Spatial Learning for Background Representation Enhancement in Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2510.18781/) (2025)
- [DMS2F-HAD: A Dual-branch Mamba-based Spatial-Spectral Fusion Network for Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2602.04102/) (2026)
- [Beyond Reconstruction: Reconstruction-to-Vector Diffusion for Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2604.11390/) (2026)

## Abstract (arXiv)

In this paper, we introduce a new approach to address the challenge of generalization in hyperspectral anomaly detection (AD). Our method eliminates the need for adjusting parameters or retraining on new test scenes as required by most existing methods. Employing an image-level training paradigm, we achieve a general anomaly enhancement network for hyperspectral AD that only needs to be trained once. Trained on a set of anomaly-free hyperspectral images with random masks, our network can learn the spatial context characteristics between anomalies and background in an unsupervised way. Additionally, a plug-and-play model selection module is proposed to search for a spatial-spectral transform domain that is more suitable for AD task than the original data. To establish a unified benchmark to comprehensively evaluate our method and existing methods, we develop a large-scale hyperspectral AD dataset (HAD100) that includes 100 real test scenes with diverse anomaly targets. In comparison experiments, we combine our network with a parameter-free detector and achieve the optimal balance between detection accuracy and inference speed among state-of-the-art AD methods. Experimental results also show that our method still achieves competitive performance when the training and test set are captured by different sensor devices. Our code is available at https://github.com/ZhaoxuLi123/AETNet.