mlsanityData to 14 Oct 2026

GEO-Bench-2: From Performance to Capability, Rethinking Evaluation in Geospatial AI

arXiv 2511.15658 ↗First posted 19 Nov 2025cs.CV cs.AIcited by 21

Not on the map. Why?

Abstract (arXiv)

Geospatial Foundation Models (GeoFMs) are transforming Earth Observation (EO), but evaluation lacks standardized protocols. GEO-Bench-2 addresses this with a comprehensive framework spanning classification, segmentation, regression, object detection, and instance segmentation across 19 permissively-licensed datasets. We introduce ''capability'' groups to rank models on datasets that share common characteristics (e.g., resolution, bands, temporality). This enables users to identify which models excel in each capability and determine which areas need improvement in future work. To support both fair comparison and methodological innovation, we define a prescriptive yet flexible evaluation protocol. This not only ensures consistency in benchmarking but also facilitates research into model adaptation strategies, a key and open challenge in advancing GeoFMs for downstream tasks. Our experiments show that no single model dominates across all tasks, confirming the specificity of the choices made during architecture design and pretraining. While models pretrained on natural images (ConvNext ImageNet, DINO V3) excel on high-resolution tasks, EO-specific models (TerraMind, Prithvi, and Clay) outperform them on multispectral applications such as agriculture and disaster response. These findings demonstrate that optimal model choice depends on task requirements, data modalities, and constraints. This shows that the goal of a single GeoFM model that performs well across all tasks remains open for future research. GEO-Bench-2 enables informed, reproducible GeoFM evaluation tailored to specific use cases. Code, data, and leaderboard for GEO-Bench-2 are publicly released under a permissive license.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Builds on 37 of its 55 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

37 of 55 references.

References · 37

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2017Mask R-CNNcited by 7,277
  3. 2016Feature Pyramid Networks for Object Detectioncited by 5,036
  4. 2022Training Compute-Optimal Large Language Modelscited by 3,019
  5. 2015Fast R-CNNcited by 2,933
  6. 2019Optuna: A Next-generation Hyperparameter Optimization Frameworkcited by 2,501
  7. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  8. 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 807
  9. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 234
  10. 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 179
  11. 2024Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applicationscited by 144
  12. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 124
25 more
  1. 2020A Generalizable and Accessible Approach to Machine Learning with Global Satellite Imagerycited by 88
  2. 2022DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentationcited by 78
  3. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 68
  4. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  5. 2021The Multi-Temporal Urban Development SpaceNet Datasetcited by 62
  6. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 53
  7. 2021Multi-Modal Temporal Attention Models for Crop Mapping from Satellite Time Seriescited by 49
  8. 2024reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysiscited by 48
  9. 2023FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagerycited by 41
  10. 2023Decoupling Common and Unique Representations for Multimodal Self-supervised Learningcited by 36
  11. 2024On the Scalability of Diffusion-based Text-to-Image Generationcited by 36
  12. 2024On the Foundations of Earth and Climate Foundation Modelscited by 33
  13. 2023FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoringcited by 31
  14. 2020ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagerycited by 27
  15. 2023Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matterscited by 27
  16. 2025TerraTorch: The Geospatial Foundation Models Toolkitcited by 25
  17. 2024Fields of The World: A Machine Learning Benchmark Dataset For Global Agricultural Field Boundary Segmentationcited by 24
  18. 2024Specialized Foundation Models Struggle to Beat Supervised Baselinescited by 24
  19. 2023There Are No Data Like More Data- Datasets for Deep Learning in Earth Observationcited by 16
  20. 2023Kuro Siwo: 33 billion m2 under the water. A global multi-temporal satellite dataset for rapid flood mappingcited by 16
  21. 2025REOBench: Benchmarking Robustness of Earth Observation Foundation Modelscited by 15
  22. 2021The SpaceNet Multi-Temporal Urban Development Challengecited by 10
  23. 2023FLAIR #2: textural and temporal information for semantic segmentation from multi-source optical imagerycited by 9
  24. 2025Fine-tune Smarter, Not Harder: Parameter-Efficient Fine-Tuning for Geospatial Foundation Modelscited by 9
  25. 2024Improving satellite imagery segmentation using multiple Sentinel-2 revisitscited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 20.

21 papers.

Sort:

By area:

  1. 2026No One Knows the State of the Art in Geospatial Foundation Modelscited by 12
  2. 2026EarthShift: a benchmark for measuring robustness to real-world distribution shifts in Earth observationcited by 11
  3. 2026Inferring Height from Earth Embeddings: First insights using Google AlphaEarthcited by 4
  4. 2026TerraFlow: Multimodal, Multitemporal Representation Learning for Earth Observationcited by 2
  5. 2026GeoNatureAgent Benchmark: Benchmarking LLM Agents for Environmental Geospatial Analysis Across Frontier and Open-Weight Foundation Modelscited by 2
  6. 2025REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agentcited by 1
  7. 2026How to Embed Matters: Evaluation of EO Embedding Design Choicescited by 1
  8. 2026Low-Rank Adaptation of Geospatial Foundation Models for Wildfire Mapping Using Sentinel-2 Datacited by 1
  9. 2026Beyond Accuracy: Assessing Calibration of Geospatial Foundation Models and Their Sensitivity to Distribution Shiftscited by 1
  10. 2026How do Self-Supervised Remote Sensing Vision Models Transfer to Downstream Tasks?cited by 0
  11. 2026Changing Modalities: Adapting Remote Sensing Models to New Satellites and Sensorscited by 0
  12. 2026Scalable and Trustworthy Earth Observation Foundation Modelscited by 0
9 more
  1. 2026Now We Know? A Systematic Comparison of TerraMind and THORcited by 0
  2. 2026Earth Observation Foundation Models for Terrestrial Ecohydrology: From Representation Learning to Process Inferencecited by 0
  3. 2026EarthVerse: Benchmarking Scientific Agents Across Dynamic Earth Systems and Natural Hazardscited by 0
  4. 2026Foundation Models Meet Agriculture: Challenges Beyond Pretrainingcited by 0
  5. 2026Infra-Bench CLS: A Global, Open-Source Benchmark for Critical Infrastructure Classification with Earth Observation Foundation Modelscited by 0
  6. 2026RSPDBench: Benchmarking Vision Foundation Models on Earth Observation Tasks Under Physically Grounded Remote-Sensing Product Degradationscited by 0
  7. 2026UltraBench 2: Towards Robust Evaluation of Vision Foundation Models on Ultrasoundcited by 0
  8. 2026Planetary Feature Fields are Scalable Earth Representationscited by 0
  9. 2026GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Taskscited by 0