# SpecAware: A Spectral-Content Aware Foundation Model for Unifying Multi-Sensor Learning in Hyperspectral Remote Sensing Mapping

arXiv: https://arxiv.org/abs/2510.27219 · HTML: https://mlsanity.com/paper/2510.27219/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 66 references that are in the map.
- First posted on arXiv 31 Oct 2025.

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks](https://mlsanity.com/paper/2106.04489/) (2021)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)
- [SpectralFormer: Rethinking Hyperspectral Image Classification with Transformers](https://mlsanity.com/paper/2107.02988/) (2021)
- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation](https://mlsanity.com/paper/2403.15356/) (2024)
- [A Brief Review of Hypernetworks in Deep Learning](https://mlsanity.com/paper/2306.06955/) (2023)
- [HyperInverter: Improving StyleGAN Inversion via Hypernetwork](https://mlsanity.com/paper/2112.00719/) (2021)
- [Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2403.05419/) (2024)
- [HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model](https://mlsanity.com/paper/2406.11519/) (2024)
- [Towards a Unified Copernicus Foundation Model for Earth Vision](https://mlsanity.com/paper/2503.11849/) (2025)
- [Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2311.15153/) (2023)
- [MambaHSI: Spatial-Spectral Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2501.04944/) (2025)
- [SpectralEarth: Training Hyperspectral Foundation Models at Scale](https://mlsanity.com/paper/2408.08447/) (2024)
- [FPGA: Fast Patch-Free Global Learning Framework for Fully End-to-End Hyperspectral Image Classification](https://mlsanity.com/paper/2011.05670/) (2020)
- [HySpecNet-11k: A Large-Scale Hyperspectral Dataset for Benchmarking Learning-Based Hyperspectral Image Compression Methods](https://mlsanity.com/paper/2306.00385/) (2023)
- [SS-MAE: Spatial-Spectral Masked Auto-Encoder for Multi-Source Remote Sensing Image Classification](https://mlsanity.com/paper/2311.04442/) (2023)
- [ABCNet: Attentive Bilateral Contextual Network for Efficient Semantic Segmentation of Fine-Resolution Remote Sensing Images](https://mlsanity.com/paper/2102.02531/) (2021)
- [HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagery](https://mlsanity.com/paper/2503.21841/) (2025)
- [There Are No Data Like More Data- Datasets for Deep Learning in Earth Observation](https://mlsanity.com/paper/2310.19231/) (2023)
- [AeroRIT: A New Scene for Hyperspectral Image Analysis](https://mlsanity.com/paper/1912.08178/) (2019)

## Built on by (most cited first, 1 of 1)

- [SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretraining](https://mlsanity.com/paper/2605.21075/) (2026)

## Abstract (arXiv)

Hyperspectral imaging (HSI) is a critical technique for fine-grained land-use and land-cover (LULC) mapping. However, the inherent heterogeneity of HSI data, particularly the variation in spectral channels across sensors, has long constrained the development of model generalization via transfer learning or joint training. Existing HSI foundation models show promise for different downstream tasks, but typically underutilize the critical guiding role of sensor meta-attributes and image semantic features, resulting in limited adaptability to cross-sensor joint learning. To address these issues, we propose SpecAware, which is a novel hyperspectral spectral-content aware foundation model for unifying multi-sensor learning for HSI mapping. To support this work, we constructed the Hyper-400K dataset, which is a new large-scale pre-training dataset with over 400,k high-quality patches from diverse airborne AVIRIS sensors that cover two data processing levels (L1 and L2). The core of SpecAware is a hypernetwork-driven unified image embedding process for HSI data. Firstly, we designed a meta-content aware module to generate a unique conditional input for each HSI sample, tailored to each spectral band by fusing the sensor meta-attributes and its own image content. Secondly, we designed the HyperEmbedding module, where a sample-conditioned hypernetwork dynamically generates a pair of matrix factors for channel-wise encoding. This process implements two-step matrix factorization, consisting of adaptive spatial pattern extraction and latent semantic feature projection, yielding a unified hyperspectral token representation. Thus, SpecAware learns to capture and interpret spatial-spectral features across diverse scenes and sensors, enabling adaptive processing of variable spectral channels within a unified multi-sensor joint pre-training framework.