mlsanityData to 7 Oct 2026

Bridging Remote Sensors with Multisensor Geospatial Foundation Models

arXiv 2404.01260 ↗First posted 1 Apr 2024cs.CV cs.AI cs.LGcited by 46

Not on the map. Why?

Abstract (arXiv)

In the realm of geospatial analysis, the diversity of remote sensors, encompassing both optical and microwave technologies, offers a wealth of distinct observational capabilities. Recognizing this, we present msGFM, a multisensor geospatial foundation model that effectively unifies data from four key sensor modalities. This integration spans an expansive dataset of two million multisensor images. msGFM is uniquely adept at handling both paired and unpaired sensor data. For data originating from identical geolocations, our model employs an innovative cross-sensor pretraining approach in masked image modeling, enabling the synthesis of joint representations from diverse sensors. msGFM, incorporating four remote sensors, upholds strong performance, forming a comprehensive model adaptable to various sensor types. msGFM has demonstrated enhanced proficiency in a range of both single-sensor and multisensor downstream tasks. These include scene classification, segmentation, cloud removal, and pan-sharpening. A key discovery of our research is that representations derived from natural images are not always compatible with the distinct characteristics of geospatial remote sensors, underscoring the limitations of existing representations in this field. Our work can serve as a guide for developing multisensor geospatial pretraining models, paving the way for more advanced geospatial capabilities.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 32 of its 64 references that are in the map. First posted on arXiv 1 Apr 2024.

Builds on

32 of 64 references.

References · 32

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  4. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  5. 2021Swin Transformer V2: Scaling Up Capacity and Resolutioncited by 1,215
  6. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  7. 2019Cross-lingual Language Model Pretrainingcited by 1,088
  8. 2021Scaling Vision with Sparse Mixture of Expertscited by 740
  9. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  10. 2022MultiMAE: Multi-modal Multi-task Masked Autoencoderscited by 287
  11. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 231
  12. 2022Tutel: Adaptive Mixture-of-Experts at Scalecited by 199
20 more
  1. 2021Muppet: Massive Multi-task Representations with Pre-Finetuningcited by 192
  2. 2020Geography-Aware Self-Supervised Learningcited by 190
  3. 2020Better Fine-Tuning by Reducing Representational Collapsecited by 156
  4. 2022An Empirical Study of Remote Sensing Pretrainingcited by 105
  5. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97
  6. 2022M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-designcited by 97
  7. 2023PreDiff: Precipitation Nowcasting with Latent Diffusion Modelscited by 96
  8. 2019SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusioncited by 95
  9. 2023Scaling Vision-Language Models with Sparse Mixture of Expertscited by 88
  10. 2021XLM-E: Cross-lingual Language Model Pre-training via ELECTRAcited by 72
  11. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  12. 2019In-domain representation learning for remote sensingcited by 41
  13. 2020Multi-Sensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagerycited by 36
  14. 2020The color out of space: learning self-supervised representations for Earth Observation imagerycited by 34
  15. 2022GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusioncited by 24
  16. 2021Heterogeneous Multi-task Learning with Expert Diversitycited by 18
  17. 2022Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learnerscited by 15
  18. 2021Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentationcited by 14
  19. 2023Lightweight In-Context Tuning for Multimodal Unified Modelscited by 4
  20. 2021On the Generalization of Agricultural Drought Classification from Climate Datacited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 24; 2026: 17.

46 papers.

Sort:

By area:

  1. 2025TerraMind: Large-Scale Generative Multimodality for Earth Observationcited by 101
  2. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 72
  3. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  4. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  5. 2024SpectralEarth: Training Hyperspectral Foundation Models at Scalecited by 42
  6. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  7. 2025Panopticon: Advancing Any-Sensor Foundation Models for Earth Observationcited by 34
  8. 2025SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensingcited by 26
  9. 2025TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observationcited by 25
  10. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  11. 2025TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Datacited by 14
  12. 2024Harnessing Massive Satellite Imagery with Efficient Masked Image Modelingcited by 12
34 more
  1. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  2. 2025Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?cited by 10
  3. 2025SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Imagescited by 9
  4. 2025TiMo: Spatiotemporal Foundation Model for Satellite Image Time Seriescited by 5
  5. 2025EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysiscited by 4
  6. 2025GeoCrossBench: Cross-Band Generalization for Remote Sensingcited by 4
  7. 2026UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observationcited by 4
  8. 2024Cross-sensor self-supervised training and alignment for remote sensingcited by 3
  9. 2025Towards Efficient Benchmarking of Foundation Models in Remote Sensing: A Capabilities Encoding Approachcited by 3
  10. 2025Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagerycited by 3
  11. 2025MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Modelscited by 3
  12. 2025Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challengescited by 3
  13. 2026TESSERA v2: Scaling Pixel-wise Earth Foundation Modelscited by 3
  14. 2025Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Datacited by 2
  15. 2025MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Datacited by 2
  16. 2026Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observationcited by 2
  17. 2026TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observationcited by 2
  18. 2025PromptMID: Modal Invariant Descriptors Based on Diffusion and Vision Foundation Models for Optical-SAR Image Matchingcited by 1
  19. 2025HiRes-FusedMIM: A High-Resolution RGB-DSM Pre-trained Model for Building-Level Remote Sensing Applicationscited by 1
  20. 2025MIMRS: A Survey on Masked Image Modeling in Remote Sensingcited by 1
  21. 2025Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challengescited by 1
  22. 2026SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensingcited by 1
  23. 2026MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applicationscited by 1
  24. 2026Pretrain Where? Investigating How Pretraining Data Diversity Impacts Geospatial Foundation Model Performancecited by 1
  25. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directionscited by 1
  26. 2025Position Prediction Self-Supervised Learning for Multimodal Satellite Imagery Semantic Segmentationcited by 0
  27. 2026Multi-Sensor Matching with HyperNetworkscited by 0
  28. 2026Unifying Heterogeneous Multi-Modal Remote Sensing Detection Via Language-Pivoted Pretrainingcited by 0
  29. 2026Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencodercited by 0
  30. 2026Earth Science Foundation Models: From Perception to Reasoning and Discoverycited by 0
  31. 2026GeoRoPE: Ground-Aware Rotary Adaptation for Remote Sensing Foundation Modelscited by 0
  32. 2026SPECTRA: Band-Routed Embedding and Stage-Wise LoRA for Cross-Sensor Fine-Tuning of Geospatial Foundation Modelscited by 0
  33. 2026Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inferencecited by 0
  34. 2026MethaneFuse: Learning from Multi-Sensor Satellite Observations for Methane Plume Detectioncited by 0