# Bridging Remote Sensors with Multisensor Geospatial Foundation Models

arXiv: https://arxiv.org/abs/2404.01260 · HTML: https://mlsanity.com/paper/2404.01260/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 64 references that are in the map.
- First posted on arXiv 1 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Cross-lingual Language Model Pretraining](https://mlsanity.com/paper/1901.07291/) (2019)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2207.08051/) (2022)
- [MultiMAE: Multi-modal Multi-task Masked Autoencoders](https://mlsanity.com/paper/2204.01678/) (2022)
- [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019)
- [Tutel: Adaptive Mixture-of-Experts at Scale](https://mlsanity.com/paper/2206.03382/) (2022)
- [Muppet: Massive Multi-task Representations with Pre-Finetuning](https://mlsanity.com/paper/2101.11038/) (2021)
- [Geography-Aware Self-Supervised Learning](https://mlsanity.com/paper/2011.09980/) (2020)
- [Better Fine-Tuning by Reducing Representational Collapse](https://mlsanity.com/paper/2008.03156/) (2020)
- [An Empirical Study of Remote Sensing Pretraining](https://mlsanity.com/paper/2204.02825/) (2022)
- [M3ViT: Mixture-of-Experts Vision Transformer for Efficient Multi-task Learning with Model-Accelerator Co-design](https://mlsanity.com/paper/2210.14793/) (2022)
- [PreDiff: Precipitation Nowcasting with Latent Diffusion Models](https://mlsanity.com/paper/2307.10422/) (2023)
- [Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIP](https://mlsanity.com/paper/2208.05516/) (2022)
- [SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusion](https://mlsanity.com/paper/1906.07789/) (2019)
- [Scaling Vision-Language Models with Sparse Mixture of Experts](https://mlsanity.com/paper/2303.07226/) (2023)
- [XLM-E: Cross-lingual Language Model Pre-training via ELECTRA](https://mlsanity.com/paper/2106.16138/) (2021)
- [Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEs](https://mlsanity.com/paper/2206.04674/) (2022)
- [In-domain representation learning for remote sensing](https://mlsanity.com/paper/1911.06721/) (2019)
- [Multi-Sensor Data Fusion for Cloud Removal in Global and All-Season Sentinel-2 Imagery](https://mlsanity.com/paper/2009.07683/) (2020)
- [The color out of space: learning self-supervised representations for Earth Observation imagery](https://mlsanity.com/paper/2006.12119/) (2020)
- [GLF-CR: SAR-Enhanced Cloud Removal with Global-Local Fusion](https://mlsanity.com/paper/2206.02850/) (2022)
- [Heterogeneous Multi-task Learning with Expert Diversity](https://mlsanity.com/paper/2106.10595/) (2021)
- [Mod-Squad: Designing Mixture of Experts As Modular Multi-Task Learners](https://mlsanity.com/paper/2212.08066/) (2022)
- [Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation](https://mlsanity.com/paper/2109.00120/) (2021)

## Built on by (most cited first, 30 of 46)

- [TerraMind: Large-Scale Generative Multimodality for Earth Observation](https://mlsanity.com/paper/2504.11171/) (2025)
- [HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Model](https://mlsanity.com/paper/2406.11519/) (2024)
- [PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models](https://mlsanity.com/paper/2412.04204/) (2024)
- [Towards a Unified Copernicus Foundation Model for Earth Vision](https://mlsanity.com/paper/2503.11849/) (2025)
- [SpectralEarth: Training Hyperspectral Foundation Models at Scale](https://mlsanity.com/paper/2408.08447/) (2024)
- [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](https://mlsanity.com/paper/2506.20380/) (2025)
- [Panopticon: Advancing Any-Sensor Foundation Models for Earth Observation](https://mlsanity.com/paper/2503.10845/) (2025)
- [SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensing](https://mlsanity.com/paper/2507.13812/) (2025)
- [TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observation](https://mlsanity.com/paper/2506.06281/) (2025)
- [A Survey on Remote Sensing Foundation Models: From Vision to Multimodality](https://mlsanity.com/paper/2503.22081/) (2025)
- [TerraMesh: A Planetary Mosaic of Multimodal Earth Observation Data](https://mlsanity.com/paper/2504.11172/) (2025)
- [Harnessing Massive Satellite Imagery with Efficient Masked Image Modeling](https://mlsanity.com/paper/2406.11933/) (2024)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [Can Generative Geospatial Diffusion Models Excel as Discriminative Geospatial Foundation Models?](https://mlsanity.com/paper/2503.07890/) (2025)
- [SMARTIES: Spectrum-Aware Multi-Sensor Auto-Encoder for Remote Sensing Images](https://mlsanity.com/paper/2506.19585/) (2025)
- [TiMo: Spatiotemporal Foundation Model for Satellite Image Time Series](https://mlsanity.com/paper/2505.08723/) (2025)
- [UniverSat: Resolution- and Modality-Agnostic Transformers for Earth Observation](https://mlsanity.com/paper/2606.23503/) (2026)
- [TESSERA v2: Scaling Pixel-wise Earth Foundation Models](https://mlsanity.com/paper/2607.03949/) (2026)
- [EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysis](https://mlsanity.com/paper/2503.15625/) (2025)
- [GeoCrossBench: Cross-Band Generalization for Remote Sensing](https://mlsanity.com/paper/2511.02831/) (2025)
- [Cross-sensor self-supervised training and alignment for remote sensing](https://mlsanity.com/paper/2405.09922/) (2024)
- [Towards Efficient Benchmarking of Foundation Models in Remote Sensing: A Capabilities Encoding Approach](https://mlsanity.com/paper/2505.03299/) (2025)
- [Flood-DamageSense: Multimodal Mamba with Multitask Learning for Building Flood Damage Assessment using SAR Remote Sensing Imagery](https://mlsanity.com/paper/2506.06667/) (2025)
- [MAPEX: Modality-Aware Pruning of Experts for Remote Sensing Foundation Models](https://mlsanity.com/paper/2507.07527/) (2025)
- [Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challenges](https://mlsanity.com/paper/2510.22964/) (2025)
- [Towards Scalable Foundation Model for Multi-modal and Hyperspectral Geospatial Data](https://mlsanity.com/paper/2503.12843/) (2025)
- [MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data](https://mlsanity.com/paper/2508.10894/) (2025)
- [Brewing Stronger Features: Dual-Teacher Distillation for Multispectral Earth Observation](https://mlsanity.com/paper/2602.19863/) (2026)
- [TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observation](https://mlsanity.com/paper/2603.12762/) (2026)
- [MOMO: Mars Orbital Model Foundation Model for Mars Orbital Applications](https://mlsanity.com/paper/2604.02719/) (2026)

## Abstract (arXiv)

In the realm of geospatial analysis, the diversity of remote sensors, encompassing both optical and microwave technologies, offers a wealth of distinct observational capabilities. Recognizing this, we present msGFM, a multisensor geospatial foundation model that effectively unifies data from four key sensor modalities. This integration spans an expansive dataset of two million multisensor images. msGFM is uniquely adept at handling both paired and unpaired sensor data. For data originating from identical geolocations, our model employs an innovative cross-sensor pretraining approach in masked image modeling, enabling the synthesis of joint representations from diverse sensors. msGFM, incorporating four remote sensors, upholds strong performance, forming a comprehensive model adaptable to various sensor types. msGFM has demonstrated enhanced proficiency in a range of both single-sensor and multisensor downstream tasks. These include scene classification, segmentation, cloud removal, and pan-sharpening. A key discovery of our research is that representations derived from natural images are not always compatible with the distinct characteristics of geospatial remote sensors, underscoring the limitations of existing representations in this field. Our work can serve as a guide for developing multisensor geospatial pretraining models, paving the way for more advanced geospatial capabilities.