mlsanityData to 7 Oct 2026

SpectralEarth: Training Hyperspectral Foundation Models at Scale

arXiv 2408.08447 ↗First posted 15 Aug 2024cs.CV cs.AIcited by 42

Not on the map. Why?

Abstract (arXiv)

Foundation models have triggered a paradigm shift in computer vision and are increasingly being adopted in remote sensing, particularly for multispectral imagery. Yet, their potential in hyperspectral imaging (HSI) remains untapped due to the absence of comprehensive and globally representative hyperspectral datasets. To close this gap, we introduce SpectralEarth, a large-scale multitemporal dataset designed to pretrain hyperspectral foundation models leveraging data from the environmental mapping and analysis program (EnMAP). SpectralEarth comprises 538 974 image patches covering 415 153 unique locations from 11 636 globally distributed EnMAP scenes spanning two years of archive. In addition, 17.5% of these locations include multiple timestamps, enabling multitemporal HSI analysis. Utilizing state-of-the-art self-supervised learning algorithms, we pretrain a series of foundation models on SpectralEarth, integrating a spectral adapter into classical vision backbones to accommodate the unique characteristics of HSI. In tandem, we construct nine downstream datasets for land-cover, crop-type mapping, and tree-species classification, providing benchmarks for model evaluation. Experimental results support the versatility of our models and their generalizability across different tasks and sensors. We also highlight computational efficiency during model fine-tuning.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 48 of its 101 references that are in the map. First posted on arXiv 15 Aug 2024.

Builds on

48 of 101 references.

References · 48

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  5. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  9. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  10. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  11. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  12. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
36 more
  1. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  2. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
  3. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  4. 2016Colorful Image Colorizationcited by 1,113
  5. 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
  6. 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 982
  7. 2019Self-Supervised Learning of Pretext-Invariant Representationscited by 806
  8. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 718
  9. 2019Self-supervised Visual Feature Learning with Deep Neural Networks: A Surveycited by 656
  10. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  11. 2018Rethinking ImageNet Pre-trainingcited by 376
  12. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  13. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  14. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 250
  15. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  16. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  17. 2022Surgical Fine-Tuning Improves Adaptation to Distribution Shiftscited by 205
  18. 2021SpectralFormer: Rethinking Hyperspectral Image Classification with Transformerscited by 168
  19. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
  20. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 143
  21. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  22. 2022VICRegL: Self-Supervised Learning of Local Visual Featurescited by 118
  23. 2020Graph Convolutional Networks for Hyperspectral Image Classificationcited by 113
  24. 2021BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrievalcited by 91
  25. 2022SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observationcited by 82
  26. 2020How Useful is Self-Supervised Pretraining for Visual Tasks?cited by 80
  27. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 72
  28. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  29. 2023Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networkscited by 55
  30. 2019Deep Learning for Classification of Hyperspectral Data: A Comparative Reviewcited by 51
  31. 2023SSL4EO-L: Datasets and Foundation Models for Landsat Imagerycited by 49
  32. 2024Bridging Remote Sensors with Multisensor Geospatial Foundation Modelscited by 46
  33. 2023HySpecNet-11k: A Large-Scale Hyperspectral Dataset for Benchmarking Learning-Based Hyperspectral Image Compression Methodscited by 26
  34. 2023Contrastive Tuning: A Little Help to Make Masked Autoencoders Forgetcited by 15
  35. 2023Toulouse Hyperspectral Data Set: a benchmark data set to assess semi-supervised spectral representation learning and pixel-wise classification techniquescited by 7
  36. 2024When are Foundation Models Effective? Understanding the Suitability for Pixel-Level Classification Using Multispectral Imagerycited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 20; 2026: 21.

42 papers.

Sort:

By area:

  1. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  2. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  3. 2025Panopticon: Advancing Any-Sensor Foundation Models for Earth Observationcited by 34
  4. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  5. 2025HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagerycited by 16
  6. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  7. 2024HyperspectralViTs: General Hyperspectral Models for On-board Remote Sensingcited by 11
  8. 2025Lossy Neural Compression for Geospatial Analytics: A Reviewcited by 10
  9. 2025Efficient Self-Supervised Learning for Earth Observation via Dynamic Dataset Curationcited by 7
  10. 2025GreenHyperSpectra: A multi-source hyperspectral dataset for global vegetation trait predictioncited by 4
  11. 2026UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observationcited by 4
  12. 2025TerraMAE: Learning Spatial-Spectral Representations from Hyperspectral Earth Observation Data via Adaptive Masked Autoencoderscited by 3
30 more
  1. 2026HyperFM: An Efficient Hyperspectral Foundation Model with Spectral Groupingcited by 3
  2. 2025Remote Sensing Semantic Segmentation Quality Assessment based on Vision Language Modelcited by 2
  3. 2025Multispectral to Hyperspectral using Pretrained Foundational modelcited by 2
  4. 2025CARL: Camera-Agnostic Representation Learning for Spectral Image Analysiscited by 2
  5. 2026Location Is All You Need: Continuous Spatiotemporal Neural Representations of Earth Observation Datacited by 2
  6. 2026SpectralEarth-FM: Bringing Hyperspectral Imagery into Multimodal Earth Observation Pretrainingcited by 2
  7. 2025A General Purpose Spectral Foundational Model for Both Proximal and Remote Sensing Spectral Imagingcited by 1
  8. 2025Parameter-Efficient Fine-Tuning of Multispectral Foundation Models for Hyperspectral Image Classificationcited by 1
  9. 2025HyBiomass: Global Hyperspectral Imagery Benchmark Dataset for Evaluating Geospatial Foundation Models in Forest Aboveground Biomass Estimationcited by 1
  10. 2025SpecAware: A Spectral-Content Aware Foundation Model for Unifying Multi-Sensor Learning in Hyperspectral Remote Sensing Mappingcited by 1
  11. 2025Leveraging AI multimodal geospatial foundation models for improved near-real-time flood mapping at a global scalecited by 1
  12. 2026Spectral Gaps and Spatial Priors: Studying Hyperspectral Downstream Adaptation Using TerraMindcited by 1
  13. 2026MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applicationscited by 1
  14. 2026TeX-1500: A Paired Real-World LWIR Hyperspectral Dataset and Benchmark for Temperature-Emissivity-Texture Decompositioncited by 1
  15. 2025Curriculum Multi-Task Self-Supervision Improves Lightweight Architectures for Onboard Satellite Hyperspectral Image Segmentationcited by 0
  16. 2025Benchmarking foundation models for hyperspectral image classification: Application to cereal crop type mappingcited by 0
  17. 2025Deep Global Clustering for Hyperspectral Image Segmentation: Concepts, Applications, and Open Challengescited by 0
  18. 2026Rethinking Test-Time Training: Tilting The Latent Distribution For Few-Shot Source-Free Adaptationcited by 0
  19. 2026Spectral-Spatial Contrastive Learning Framework for Regression on Hyperspectral Datacited by 0
  20. 2026SSFT: A Lightweight Spectral-Spatial Fusion Transformer for Generic Hyperspectral Classificationcited by 0
  21. 2026Cross-Domain Transfer of Hyperspectral Foundation Modelscited by 0
  22. 2026Foundation AI Models for Aerosol Optical Depth Estimation from PACE Satellite Datacited by 0
  23. 2026Earth Science Foundation Models: From Perception to Reasoning and Discoverycited by 0
  24. 2026ChronoEarth-492K: A Large Scale and Long Horizon Spatiotemporal Hyperspectral Earth Observation Dataset and Benchmarkcited by 0
  25. 2026LESSViT: Robust Hyperspectral Representation Learning under Spectral Configuration Shiftcited by 0
  26. 2026Self-supervised training for high-resolution close-range multispectral remote sensing imagerycited by 0
  27. 2026MBTI: A Multi-Branch Efficient Fine-Tuning Framework for Hyperspectral Image Classification with Foundation Modelscited by 0
  28. 2026Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inferencecited by 0
  29. 2026Beyond Accuracy: Assessing Calibration of Geospatial Foundation Models and Their Sensitivity to Distribution Shiftscited by 0
  30. 2026HyperSAM: A Promptable Foundation Model for Hyperspectral Remote Sensingcited by 0