# Multimodal contrastive learning for remote sensing tasks

arXiv: https://arxiv.org/abs/2209.02329 · HTML: https://mlsanity.com/paper/2209.02329/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 6 Sep 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [Large Batch Training of Convolutional Networks](https://mlsanity.com/paper/1708.03888/) (2017)
- [Rethinking ImageNet Pre-training](https://mlsanity.com/paper/1811.08883/) (2018)
- [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021)
- [Geography-Aware Self-Supervised Learning](https://mlsanity.com/paper/2011.09980/) (2020)
- [Demystifying Contrastive Self-Supervised Learning: Invariances, Augmentations and Dataset Biases](https://mlsanity.com/paper/2007.13916/) (2020)
- [When Does Contrastive Visual Representation Learning Work?](https://mlsanity.com/paper/2105.05837/) (2021)
- [Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding](https://mlsanity.com/paper/2104.07070/) (2021)
- [Unsupervised Feature Learning by Cross-Level Instance-Group Discrimination](https://mlsanity.com/paper/2008.03813/) (2020)
- [Revisiting Contrastive Methods for Unsupervised Learning of Visual Representations](https://mlsanity.com/paper/2106.05967/) (2021)
- [Self-supervised Change Detection in Multi-view Remote Sensing Images](https://mlsanity.com/paper/2103.05969/) (2021)
- [Self-supervised Audiovisual Representation Learning for Remote Sensing Data](https://mlsanity.com/paper/2108.00688/) (2021)
- [Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Images](https://mlsanity.com/paper/2205.02049/) (2022)
- [Self-Supervised Multisensor Change Detection](https://mlsanity.com/paper/2103.05102/) (2021)
- [Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentation](https://mlsanity.com/paper/2109.00120/) (2021)
- [Predicting Economic Development using Geolocated Wikipedia Articles](https://mlsanity.com/paper/1905.01627/) (2019)
- [Self-supervised Remote Sensing Images Change Detection at Pixel-level](https://mlsanity.com/paper/2105.08501/) (2021)

## Built on by (most cited first, 18 of 18)

- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [A Billion-scale Foundation Model for Remote Sensing Images](https://mlsanity.com/paper/2304.05215/) (2023)
- [FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space](https://mlsanity.com/paper/2310.20071/) (2023)
- [USat: A Unified Self-Supervised Encoder for Multi-Sensor Satellite Imagery](https://mlsanity.com/paper/2312.02199/) (2023)
- [RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation](https://mlsanity.com/paper/2504.03166/) (2025)
- [OpenForest: A data catalogue for machine learning in forest monitoring](https://mlsanity.com/paper/2311.00277/) (2023)
- [GAIR: Location-Aware Self-Supervised Contrastive Pre-Training with Geo-Aligned Implicit Representations](https://mlsanity.com/paper/2503.16683/) (2025)
- [Bootstrapping Autonomous Driving Radars with Self-Supervised Learning](https://mlsanity.com/paper/2312.04519/) (2023)
- [EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysis](https://mlsanity.com/paper/2503.15625/) (2025)
- [A Genealogy of Foundation Models in Remote Sensing](https://mlsanity.com/paper/2504.17177/) (2025)
- [Zooming Out on Zooming In: Advancing Super-Resolution for Remote Sensing](https://mlsanity.com/paper/2311.18082/) (2023)
- [On the Importance of Feature Representation for Flood Mapping using Classical Machine Learning Approaches](https://mlsanity.com/paper/2303.00691/) (2023)
- [MoSAiC: Multi-Modal Multi-Label Supervision-Aware Contrastive Learning for Remote Sensing](https://mlsanity.com/paper/2507.08683/) (2025)
- [ZAYAN: Disentangled Contrastive Transformer for Tabular Remote Sensing Data](https://mlsanity.com/paper/2604.27606/) (2026)
- [FACTUAL: A Novel Framework for Contrastive Learning Based Robust SAR Image Classification](https://mlsanity.com/paper/2404.03225/) (2024)
- [Rapid Wildfire Hotspot Detection Using Self-Supervised Learning on Temporal Remote Sensing Data](https://mlsanity.com/paper/2405.20093/) (2024)
- [Can multimodal representation learning by alignment preserve modality-specific information?](https://mlsanity.com/paper/2509.17943/) (2025)
- [Pixel-Wise Multimodal Contrastive Learning for Remote Sensing Images](https://mlsanity.com/paper/2601.04127/) (2026)

## Abstract (arXiv)

Self-supervised methods have shown tremendous success in the field of computer vision, including applications in remote sensing and medical imaging. Most popular contrastive-loss based methods like SimCLR, MoCo, MoCo-v2 use multiple views of the same image by applying contrived augmentations on the image to create positive pairs and contrast them with negative examples. Although these techniques work well, most of these techniques have been tuned on ImageNet (and similar computer vision datasets). While there have been some attempts to capture a richer set of deformations in the positive samples, in this work, we explore a promising alternative to generating positive examples for remote sensing data within the contrastive learning framework. Images captured from different sensors at the same location and nearby timestamps can be thought of as strongly augmented instances of the same scene, thus removing the need to explore and tune a set of hand crafted strong augmentations. In this paper, we propose a simple dual-encoder framework, which is pre-trained on a large unlabeled dataset (~1M) of Sentinel-1 and Sentinel-2 image pairs. We test the embeddings on two remote sensing downstream tasks: flood segmentation and land cover mapping, and empirically show that embeddings learnt from this technique outperform the conventional technique of collecting positive examples via aggressive data augmentations.