# The color out of space: learning self-supervised representations for Earth Observation imagery

arXiv: https://arxiv.org/abs/2006.12119 · HTML: https://mlsanity.com/paper/2006.12119/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 30 of 34)

- [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021)
- [CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoders](https://mlsanity.com/paper/2311.00566/) (2023)
- [RS5M and GeoRSCLIP: A Large Scale Vision-Language Dataset and A Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2306.11300/) (2023)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [Towards Geospatial Foundation Models via Continual Pretraining](https://mlsanity.com/paper/2302.04476/) (2023)
- [An Empirical Study of Remote Sensing Pretraining](https://mlsanity.com/paper/2204.02825/) (2022)
- [Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding](https://mlsanity.com/paper/2104.07070/) (2021)
- [Bridging Remote Sensors with Multisensor Geospatial Foundation Models](https://mlsanity.com/paper/2404.01260/) (2024)
- [Geographical Knowledge-driven Representation Learning for Remote Sensing Images](https://mlsanity.com/paper/2107.05276/) (2021)
- [FCCDN: Feature Constraint Network for VHR Image Change Detection](https://mlsanity.com/paper/2105.10860/) (2021)
- [Self-supervised Audiovisual Representation Learning for Remote Sensing Data](https://mlsanity.com/paper/2108.00688/) (2021)
- [Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works](https://mlsanity.com/paper/2211.08129/) (2022)
- [Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation](https://mlsanity.com/paper/2205.11319/) (2022)
- [Self-Supervised Learning for Invariant Representations from Multi-Spectral and SAR Images](https://mlsanity.com/paper/2205.02049/) (2022)
- [Extending global-local view alignment for self-supervised learning with remote sensing imagery](https://mlsanity.com/paper/2303.06670/) (2023)
- [DDPM-CD: Denoising Diffusion Probabilistic Models as Feature Extractors for Change Detection](https://mlsanity.com/paper/2206.11892/) (2022)
- [Benchmarking and scaling of deep learning models for land cover image classification](https://mlsanity.com/paper/2111.09451/) (2021)
- [Mapping smallholder cashew plantations to inform sustainable tree crop expansion in Benin](https://mlsanity.com/paper/2301.00363/) (2023)
- [CtxMIM: Context-Enhanced Masked Image Modeling for Remote Sensing Image Understanding](https://mlsanity.com/paper/2310.00022/) (2023)
- [Evaluating Self and Semi-Supervised Methods for Remote Sensing Segmentation Tasks](https://mlsanity.com/paper/2111.10079/) (2021)
- [Clustering augmented Self-Supervised Learning: Anapplication to Land Cover Mapping](https://mlsanity.com/paper/2108.07323/) (2021)
- [C-SAW: Self-Supervised Prompt Learning for Image Generalization in Remote Sensing](https://mlsanity.com/paper/2311.15812/) (2023)
- [Semi-supervised learning for joint SAR and multispectral land cover classification](https://mlsanity.com/paper/2108.09075/) (2021)
- [Self-Supervised In-Domain Representation Learning for Remote Sensing Image Scene Classification](https://mlsanity.com/paper/2302.01793/) (2023)
- [Land Cover Mapping in Limited Labels Scenario: A Survey](https://mlsanity.com/paper/2103.02429/) (2021)
- [A Moment in the Sun: Solar Nowcasting from Multispectral Satellite Data using Self-Supervised Learning](https://mlsanity.com/paper/2112.13974/) (2021)
- [Cross-sensor self-supervised training and alignment for remote sensing](https://mlsanity.com/paper/2405.09922/) (2024)
- [Terrain-Informed Self-Supervised Learning: Enhancing Building Footprint Extraction from LiDAR Data with Limited Annotations](https://mlsanity.com/paper/2311.01188/) (2023)
- [Time2Agri: Temporal Pretext Tasks for Agricultural Monitoring](https://mlsanity.com/paper/2507.04366/) (2025)
- [Spotting Virus from Satellites: Modeling the Circulation of West Nile Virus Through Graph Neural Networks](https://mlsanity.com/paper/2209.05251/) (2022)

## Abstract (arXiv)

The recent growth in the number of satellite images fosters the development of effective deep-learning techniques for Remote Sensing (RS). However, their full potential is untapped due to the lack of large annotated datasets. Such a problem is usually countered by fine-tuning a feature extractor that is previously trained on the ImageNet dataset. Unfortunately, the domain of natural images differs from the RS one, which hinders the final performance. In this work, we propose to learn meaningful representations from satellite imagery, leveraging its high-dimensionality spectral bands to reconstruct the visible colors. We conduct experiments on land cover classification (BigEarthNet) and West Nile Virus detection, showing that colorization is a solid pretext task for training a feature extractor. Furthermore, we qualitatively observe that guesses based on natural images and colorization rely on different parts of the input. This paves the way to an ensemble model that eventually outperforms both the above-mentioned techniques.