# EarthNets: Empowering AI in Earth Observation

arXiv: https://arxiv.org/abs/2210.04936 · HTML: https://mlsanity.com/paper/2210.04936/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 155 of its 427 references that are in the map.
- First posted on arXiv 10 Oct 2022.

## Builds on (most cited first, 30 of 155)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Improved Baselines with Momentum Contrastive Learning](https://mlsanity.com/paper/2003.04297/) (2020)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [MLP-Mixer: An all-MLP Architecture for Vision](https://mlsanity.com/paper/2105.01601/) (2021)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Remote Sensing Image Scene Classification: Benchmark and State of the Art](https://mlsanity.com/paper/1703.00121/) (2017)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classification](https://mlsanity.com/paper/1608.05167/) (2016)

## Built on by (most cited first, 30 of 40)

- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Model](https://mlsanity.com/paper/2401.09712/) (2024)
- [Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observation](https://mlsanity.com/paper/2403.15356/) (2024)
- [Vision-Language Models in Remote Sensing: Current Progress and Future Trends](https://mlsanity.com/paper/2305.05726/) (2023)
- [SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observation](https://mlsanity.com/paper/2211.07044/) (2022)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing](https://mlsanity.com/paper/2310.18653/) (2023)
- [Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents](https://mlsanity.com/paper/2509.23141/) (2025)
- [One for All: Toward Unified Foundation Models for Earth Vision](https://mlsanity.com/paper/2401.07527/) (2024)
- [On the Foundations of Earth and Climate Foundation Models](https://mlsanity.com/paper/2405.04285/) (2024)
- [TorchSpatial: A Location Encoding Framework and Benchmark for Spatial Representation Learning](https://mlsanity.com/paper/2406.15658/) (2024)
- [Parameter-Efficient Transfer Learning for Remote Sensing Image-Text Retrieval](https://mlsanity.com/paper/2308.12509/) (2023)
- [ChatEarthNet: A Global-Scale Image-Text Dataset Empowering Vision-Language Geo-Foundation Models](https://mlsanity.com/paper/2402.11325/) (2024)
- [Multi-modal learning for geospatial vegetation forecasting](https://mlsanity.com/paper/2303.16198/) (2023)
- [There Are No Data Like More Data- Datasets for Deep Learning in Earth Observation](https://mlsanity.com/paper/2310.19231/) (2023)
- [SATIN: A Multi-Task Metadataset for Classifying Satellite Imagery using Vision-Language Models](https://mlsanity.com/paper/2304.11619/) (2023)
- [UrbanSARFloods: Sentinel-1 SLC-Based Benchmark Dataset for Urban and Open-Area Flood Mapping](https://mlsanity.com/paper/2406.04111/) (2024)
- [EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation](https://mlsanity.com/paper/2605.29330/) (2026)
- [HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images](https://mlsanity.com/paper/2309.16486/) (2023)
- [Multi-Modal and Multi-Resolution Data Fusion for High-Resolution Cloud Removal: A Novel Baseline and Benchmark](https://mlsanity.com/paper/2301.03432/) (2023)
- [Multilingual Augmentation for Robust Visual Question Answering in Remote Sensing Images](https://mlsanity.com/paper/2304.03844/) (2023)
- [Multi-Spectral Remote Sensing Image Retrieval Using Geospatial Foundation Models](https://mlsanity.com/paper/2403.02059/) (2024)
- [Few-shot Object Detection in Remote Sensing: Lifting the Curse of Incompletely Annotated Novel Objects](https://mlsanity.com/paper/2309.10588/) (2023)
- [MultiMAE Meets Earth Observation: Pre-training Multi-modal Multi-task Masked Autoencoders for Earth Observation Tasks](https://mlsanity.com/paper/2505.14951/) (2025)
- [GAMUS: A Geometry-aware Multi-modal Semantic Segmentation Benchmark for Remote Sensing Data](https://mlsanity.com/paper/2305.14914/) (2023)
- [When Geoscience Meets Generative AI and Large Language Models: Foundations, Trends, and Future Challenges](https://mlsanity.com/paper/2402.03349/) (2024)
- [Urban Flood Mapping Using Satellite Synthetic Aperture Radar Data: A Review of Characteristics, Approaches and Datasets](https://mlsanity.com/paper/2411.04153/) (2024)
- [Exploring Geometric Deep Learning For Precipitation Nowcasting](https://mlsanity.com/paper/2309.05828/) (2023)

## Abstract (arXiv)

Earth observation (EO), aiming at monitoring the state of planet Earth using remote sensing data, is critical for improving our daily lives and living environment. With a growing number of satellites in orbit, an increasing number of datasets with diverse sensors and research domains are being published to facilitate the research of the remote sensing community. This paper presents a comprehensive review of more than 500 publicly published datasets, including research domains like agriculture, land use and land cover, disaster monitoring, scene understanding, vision-language models, foundation models, climate change, and weather forecasting. We systematically analyze these EO datasets from four aspects: volume, resolution distributions, research domains, and the correlation between datasets. Based on the dataset attributes, we propose to measure, rank, and select datasets to build a new benchmark for model evaluation. Furthermore, a new platform for EO, termed EarthNets, is released to achieve a fair and consistent evaluation of deep learning methods on remote sensing data. EarthNets supports standard dataset libraries and cutting-edge deep learning models to bridge the gap between the remote sensing and machine learning communities. Based on this platform, extensive deep-learning methods are evaluated on the new benchmark. The insightful results are beneficial to future research. The platform and dataset collections are publicly available at https://earthnets.github.io.