mlsanityData to 7 Oct 2026

SpectralGPT: Spectral Remote Sensing Foundation Model

arXiv 2311.07113 ↗First posted 13 Nov 2023cs.CVcited by 250

Not on the map. Why?

Abstract (arXiv)

The foundation model has recently garnered significant attention due to its potential to revolutionize the field of visual representation learning in a self-supervised manner. While most foundation models are tailored to effectively process RGB images for various visual tasks, there is a noticeable gap in research focused on spectral data, which offers valuable information for scene understanding, especially in remote sensing (RS) applications. To fill this gap, we created for the first time a universal RS foundation model, named SpectralGPT, which is purpose-built to handle spectral RS images using a novel 3D generative pretrained transformer (GPT). Compared to existing foundation models, SpectralGPT 1) accommodates input images with varying sizes, resolutions, time series, and regions in a progressive training fashion, enabling full utilization of extensive RS big data; 2) leverages 3D token generation for spatial-spectral coupling; 3) captures spectrally sequential patterns via multi-target reconstruction; 4) trains on one million spectral RS images, yielding models with over 600 million parameters. Our evaluation highlights significant performance improvements with pretrained SpectralGPT models, signifying substantial potential in advancing spectral RS big data applications within the field of geoscience across four downstream tasks: single/multi-label scene classification, semantic segmentation, and change detection.

Key facts. Cited by 250 papers in the map (data through 7 Oct 2026). Builds on 32 of its 54 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

32 of 54 references.

References · 32

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  12. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
20 more
  1. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  2. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  3. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  4. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  5. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  6. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  7. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  8. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  9. 2017Functional Map of the Worldcited by 338
  10. 2020Propagate Yourself: Exploring Pixel-Level Consistency for Unsupervised Visual Representation Learningcited by 297
  11. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  12. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 214
  13. 2022Advancing Plain Vision Transformer Towards Remote Sensing Foundation Modelcited by 149
  14. 2018Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networkscited by 108
  15. 2020LoCo: Local Contrastive Representation Learningcited by 55
  16. 2023Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networkscited by 55
  17. 2020Non-local Meets Global: An Iterative Paradigm for Hyperspectral Image Restorationcited by 43
  18. 2019In-domain representation learning for remote sensingcited by 41
  19. 2022Recent Advances for Quantum Neural Networks in Generative Learningcited by 39
  20. 2022Decoupled-and-Coupled Networks: Self-Supervised Hyperspectral Image Super-Resolution with Subpixel Fusioncited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 62; 2025: 115; 2026: 68.

250 papers.

Sort:

By area:

  1. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 97
  2. 2024AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalitiescited by 73
  3. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 72
  4. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  5. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  6. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  7. 2024SpectralEarth: Training Hyperspectral Foundation Models at Scalecited by 42
  8. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  9. 2025BRIGHT: A globally distributed multimodal building damage assessment dataset with very-high-resolution for all-weather disaster responsecited by 39
  10. 2024Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Modelcited by 37
  11. 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 37
  12. 2024SpectralMamba: Efficient Mamba for Hyperspectral Image Classificationcited by 36
38 more
  1. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  2. 2023SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraintscited by 35
  3. 2025Pulse-PPG: An Open-Source Field-Trained PPG Foundation Model for Wearable Applications Across Lab and Field Settingscited by 35
  4. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 35
  5. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  6. 2023FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoringcited by 30
  7. 2024Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretrainingcited by 27
  8. 2024CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentationcited by 27
  9. 2023Time Travelling Pixels: Bitemporal Features Integration with Foundation Model for Remote Sensing Image Change Detectioncited by 26
  10. 2025SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensingcited by 26
  11. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  12. 2024SegEarth-OV: Towards Training-Free Open-Vocabulary Segmentation for Remote Sensing Imagescited by 25
  13. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  14. 2023Better, Not Just More: Data-Centric Machine Learning for Earth Observationcited by 19
  15. 2024A Comprehensive Survey for Hyperspectral Image Classification: The Evolution from Conventional to Transformers and Mamba Modelscited by 19
  16. 2024How to Learn More? Exploring Kolmogorov-Arnold Networks for Hyperspectral Image Classificationcited by 19
  17. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  18. 2025RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Modelscited by 17
  19. 2025HyperFree: A Channel-adaptive and Tuning-free Foundation Model for Hyperspectral Remote Sensing Imagerycited by 16
  20. 2025RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretationcited by 16
  21. 2024Triple-domain Feature Learning with Frequency-aware Memory Enhancement for Moving Infrared Small Target Detectioncited by 15
  22. 2024RS-MoE: A Vision-Language Model with Mixture of Experts for Remote Sensing Image Captioning and Visual Question Answeringcited by 15
  23. 2025A Token-level Text Image Foundation Model for Document Understandingcited by 13
  24. 2025GIScience in the Era of Artificial Intelligence: A Research Agenda Towards Autonomous GIScited by 13
  25. 2025REOBench: Benchmarking Robustness of Earth Observation Foundation Modelscited by 13
  26. 2024Decomposition-based Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentationcited by 12
  27. 2024Harnessing Massive Satellite Imagery with Efficient Masked Image Modelingcited by 12
  28. 2025ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wildcited by 12
  29. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  30. 2024LiDAR-Guided Cross-Attention Fusion for Hyperspectral Band Selection and Image Classificationcited by 11
  31. 2024Pyramid Hierarchical Transformer for Hyperspectral Image Classificationcited by 11
  32. 2024Towards Natural Machine Unlearningcited by 11
  33. 2024WaveMamba: Spatial-Spectral Wavelet Mamba for Hyperspectral Image Classificationcited by 11
  34. 2024OReole-FM: successes and challenges toward billion-parameter foundation models for high-resolution satellite imagerycited by 11
  35. 2024RS-vHeat: Heat Conduction Guided Efficient Remote Sensing Foundation Modelcited by 11
  36. 2025FedRSClip: Federated Learning for Remote Sensing Scene Classification Using Vision-Language Modelscited by 11
  37. 2025RoMA: Scaling up Mamba-based Foundation Models for Remote Sensingcited by 11
  38. 2024Siamese Meets Diffusion Network: SMDNet for Enhanced Change Detection in High-Resolution RS Imagerycited by 10

50 most cited of 250 All 250 (JSON)