# Learning a Cross-modality Anomaly Detector for Remote Sensing Imagery

arXiv: https://arxiv.org/abs/2310.07511 · HTML: https://mlsanity.com/paper/2310.07511/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 84 references that are in the map.
- First posted on arXiv 11 Oct 2023.

## Builds on (most cited first, 8 of 8)

- [A Unified Model for Multi-class Anomaly Detection](https://mlsanity.com/paper/2206.03687/) (2022)
- [Effective Fusion Factor in FPN for Tiny Object Detection](https://mlsanity.com/paper/2011.02298/) (2020)
- [An Alternative Cross Entropy Loss for Learning-to-Rank](https://mlsanity.com/paper/1911.09798/) (2019)
- [Efficient Optimization for Rank-based Loss Functions](https://mlsanity.com/paper/1604.08269/) (2016)
- [Graph Regularized Autoencoder and its Application in Unsupervised Anomaly Detection](https://mlsanity.com/paper/2010.15949/) (2020)
- [GAN-based Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2007.02441/) (2020)
- [Anomaly Segmentation for High-Resolution Remote Sensing Images Based on Pixel Descriptors](https://mlsanity.com/paper/2301.13422/) (2023)
- [One-Step Detection Paradigm for Hyperspectral Anomaly Detection via Spectral Deviation Relationship Learning](https://mlsanity.com/paper/2303.12342/) (2023)

## Built on by (most cited first, 5 of 5)

- [HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model](https://mlsanity.com/paper/2406.11519/) (2024)
- [M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection](https://mlsanity.com/paper/2505.10931/) (2025)
- [A Survey on Foundation-Model-Based Industrial Defect Detection](https://mlsanity.com/paper/2502.19106/) (2025)
- [Overcoming the Identity Mapping Problem in Self-Supervised Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2504.04115/) (2025)
- [Effective Receptive Field Ordering Matters for Infrared Small Target Detection](https://mlsanity.com/paper/2607.23994/) (2026)

## Abstract (arXiv)

Remote sensing anomaly detector can find the objects deviating from the background as potential targets for Earth monitoring. Given the diversity in earth anomaly types, designing a transferring model with cross-modality detection ability should be cost-effective and flexible to new earth observation sources and anomaly types. However, the current anomaly detectors aim to learn the certain background distribution, the trained model cannot be transferred to unseen images. Inspired by the fact that the deviation metric for score ranking is consistent and independent from the image distribution, this study exploits the learning target conversion from the varying background distribution to the consistent deviation metric. We theoretically prove that the large-margin condition in labeled samples ensures the transferring ability of learned deviation metric. To satisfy this condition, two large margin losses for pixel-level and feature-level deviation ranking are proposed respectively. Since the real anomalies are difficult to acquire, anomaly simulation strategies are designed to compute the model loss. With the large-margin learning for deviation metric, the trained model achieves cross-modality detection ability in five modalities including hyperspectral, visible light, synthetic aperture radar (SAR), infrared and low-light in zero-shot manner.