Decoupled Complementary Spectral-Spatial Learning for Background Representation Enhancement in Hyperspectral Anomaly Detection

A recent class of hyperspectral anomaly detection methods can be trained once on background datasets and then deployed universally without per-scene retraining or parameter tuning, showing strong efficiency and robustness. Building upon this paradigm, we propose a decoupled complementary spectral--spatial learning framework for background representation enhancement. The framework follows a two-stage training strategy: (1) we first train a spectral enhancement network via reverse distillation to obtain robust background spectral representations; and (2) we then freeze the spectral branch as a teacher and train a spatial branch as a complementary student (the "rebellious student") to capture spatial patterns overlooked by the teacher. Complementary learning is achieved through decorrelation objectives that reduce representational redundancy between the two branches, together with reconstruction regularization to prevent the student from learning irrelevant noise. After training, the framework jointly enhances background representations from both spectral and spatial perspectives, and the resulting enhanced features can be plugged into parameter-free, training-free detectors (e.g., the Reed--Xiaoli (RX) detector) for test-time deployment without per-scene retraining or parameter tuning. Experiments on the HAD100 benchmark demonstrate substantial improvements over representative baselines with modest computational overhead, validating the effectiveness of the proposed complementary learning paradigm. Our code is publicly available at https://github.com/xjpp2016/FERS.