# GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI

arXiv: https://arxiv.org/abs/2511.15658 · HTML: https://mlsanity.com/paper/2511.15658/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 19 Nov 2025.

## Builds on (most cited first, 30 of 37)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classification](https://mlsanity.com/paper/1709.00029/) (2017)
- [Deep Reinforcement Learning at the Edge of the Statistical Precipice](https://mlsanity.com/paper/2108.13264/) (2021)
- [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applications](https://mlsanity.com/paper/2412.02732/) (2024)
- [GEO-Bench: Toward Foundation Models for Earth Monitoring](https://mlsanity.com/paper/2306.03831/) (2023)
- [A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagery](https://mlsanity.com/paper/2010.08168/) (2020)
- [DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentation](https://mlsanity.com/paper/2203.12560/) (2022)
- [PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Models](https://mlsanity.com/paper/2412.04204/) (2024)
- [OmniSat: Self-Supervised Modality Fusion for Earth Observation](https://mlsanity.com/paper/2404.08351/) (2024)
- [The Multi-Temporal Urban Development SpaceNet Dataset](https://mlsanity.com/paper/2102.04420/) (2021)
- [Towards a Unified Copernicus Foundation Model for Earth Vision](https://mlsanity.com/paper/2503.11849/) (2025)
- [Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Series](https://mlsanity.com/paper/2112.07558/) (2021)
- [reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysis](https://mlsanity.com/paper/2407.03653/) (2024)
- [FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery](https://mlsanity.com/paper/2310.13336/) (2023)
- [Decoupling Common and Unique Representations for Multimodal Self-supervised Learning](https://mlsanity.com/paper/2309.05300/) (2023)
- [On the Scalability of Diffusion-based Text-to-Image Generation](https://mlsanity.com/paper/2404.02883/) (2024)
- [On the Foundations of Earth and Climate Foundation Models](https://mlsanity.com/paper/2405.04285/) (2024)
- [FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoring](https://mlsanity.com/paper/2312.10114/) (2023)
- [ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagery](https://mlsanity.com/paper/2011.05479/) (2020)
- [Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matters](https://mlsanity.com/paper/2305.13456/) (2023)
- [TerraTorch: The Geospatial Foundation Models Toolkit](https://mlsanity.com/paper/2503.20563/) (2025)
- [Fields of The World: A Machine Learning Benchmark Dataset For Global Agricultural Field Boundary Segmentation](https://mlsanity.com/paper/2409.16252/) (2024)
- [Specialized Foundation Models Struggle to Beat Supervised Baselines](https://mlsanity.com/paper/2411.02796/) (2024)

## Built on by (most cited first, 21 of 21)

- [No One Knows the State of the Art in Geospatial Foundation Models](https://mlsanity.com/paper/2605.12678/) (2026)
- [EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observation](https://mlsanity.com/paper/2605.29330/) (2026)
- [Inferring Height from Earth Embeddings: First insights using Google AlphaEarth](https://mlsanity.com/paper/2602.17250/) (2026)
- [TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observation](https://mlsanity.com/paper/2603.12762/) (2026)
- [GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Models](https://mlsanity.com/paper/2606.12821/) (2026)
- [REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agent](https://mlsanity.com/paper/2511.17442/) (2025)
- [How to Embed Matters: Evaluation of EO Embedding Design Choices](https://mlsanity.com/paper/2603.10658/) (2026)
- [Low-Rank Adaptation of Geospatial Foundation Models for Wildfire Mapping Using Sentinel-2 Data](https://mlsanity.com/paper/2605.04989/) (2026)
- [Beyond Accuracy: Assessing Calibration of Geospatial Foundation Models and Their Sensitivity to Distribution Shifts](https://mlsanity.com/paper/2608.16614/) (2026)
- [How do Self-Supervised Remote Sensing Vision Models Transfer to Downstream Tasks?](https://mlsanity.com/paper/2606.13896/) (2026)
- [Changing Modalities: Adapting Remote Sensing Models to New Satellites and Sensors](https://mlsanity.com/paper/2606.23356/) (2026)
- [Scalable and Trustworthy Earth Observation Foundation Models](https://mlsanity.com/paper/2607.07758/) (2026)
- [Now We Know? A Systematic Comparison of TerraMind and THOR](https://mlsanity.com/paper/2607.18504/) (2026)
- [Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inference](https://mlsanity.com/paper/2608.15282/) (2026)
- [EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazards](https://mlsanity.com/paper/2608.23525/) (2026)
- [Foundation Models Meet Agriculture: Challenges Beyond Pretraining](https://mlsanity.com/paper/2608.30392/) (2026)
- [Infra-Bench CLS: A Global, Open-Source Benchmark for Critical Infrastructure Classification with Earth Observation Foundation Models](https://mlsanity.com/paper/2609.09482/) (2026)
- [RSPDBench: Benchmarking Vision Foundation Models on Earth Observation Tasks Under Physically Grounded Remote-Sensing Product Degradations](https://mlsanity.com/paper/2609.23427/) (2026)
- [UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasound](https://mlsanity.com/paper/2609.28610/) (2026)
- [Planetary Feature Fields are Scalable Earth Representations](https://mlsanity.com/paper/2609.37784/) (2026)
- [GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks](https://mlsanity.com/paper/2610.09112/) (2026)

## Abstract (arXiv)

Geospatial Foundation Models (GeoFMs) are transforming Earth Observation (EO), but evaluation lacks standardized protocols. GEO-Bench-2 addresses this with a comprehensive framework spanning classification, segmentation, regression, object detection, and instance segmentation across 19 permissively-licensed datasets. We introduce ''capability'' groups to rank models on datasets that share common characteristics (e.g., resolution, bands, temporality). This enables users to identify which models excel in each capability and determine which areas need improvement in future work. To support both fair comparison and methodological innovation, we define a prescriptive yet flexible evaluation protocol. This not only ensures consistency in benchmarking but also facilitates research into model adaptation strategies, a key and open challenge in advancing GeoFMs for downstream tasks. Our experiments show that no single model dominates across all tasks, confirming the specificity of the choices made during architecture design and pretraining. While models pretrained on natural images (ConvNext ImageNet, DINO V3) excel on high-resolution tasks, EO-specific models (TerraMind, Prithvi, and Clay) outperform them on multispectral applications such as agriculture and disaster response. These findings demonstrate that optimal model choice depends on task requirements, data modalities, and constraints. This shows that the goal of a single GeoFM model that performs well across all tasks remains open for future research. GEO-Bench-2 enables informed, reproducible GeoFM evaluation tailored to specific use cases. Code, data, and leaderboard for GEO-Bench-2 are publicly released under a permissive license.