mlsanityData to 14 Oct 2026

Towards Efficient Benchmarking of Foundation Models in Remote Sensing: A Capabilities Encoding Approach

arXiv 2505.03299 ↗First posted 6 May 2025cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Foundation models constitute a significant advancement in computer vision: after a single, albeit costly, training phase, they can address a wide array of tasks. In the field of Earth observation, over 75 remote sensing vision foundation models have been developed in the past four years. However, none has consistently outperformed the others across all available downstream tasks. To facilitate their comparison, we propose a cost-effective method for predicting a model's performance on multiple downstream tasks without the need for fine-tuning on each one. This method is based on what we call "capabilities encoding." The utility of this novel approach is twofold: we demonstrate its potential to simplify the selection of a foundation model for a given new task, and we employ it to offer a fresh perspective on the existing literature, suggesting avenues for future research. Codes are available at https://github.com/pierreadorni/capabilities-encoding.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 51 of its 80 references that are in the map. First posted on arXiv 6 May 2025.

Builds on

51 of 80 references.

References · 51

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,750
  6. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  7. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  8. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  9. 2016AID: A Benchmark Dataset for Performance Evaluation of Aerial Scene Classificationcited by 347
  10. 2019Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmarkcited by 340
  11. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  12. 2018DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Imagescited by 277
38 more
  1. 2021LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentationcited by 261
  2. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
  3. 2019BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understandingcited by 234
  4. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 215
  5. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
  6. 2021Object Detection in Aerial Images: A Large-Scale Benchmark and Challengescited by 204
  7. 2019iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Imagescited by 197
  8. 2020Geography-Aware Self-Supervised Learningcited by 190
  9. 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 179
  10. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 169
  11. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 167
  12. 2023SatCLIP: Global, General-Purpose Location Embeddings with Satellite Imagerycited by 151
  13. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 149
  14. 2018xView: Objects in Context in Overhead Imagerycited by 146
  15. 2021FAIR1M: A Benchmark Dataset for Fine-grained Object Recognition in High-Resolution Remote Sensing Imagerycited by 143
  16. 2023GEO-Bench: Toward Foundation Models for Earth Monitoringcited by 124
  17. 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 119
  18. 2017PatternNet: A Benchmark Dataset for Performance Evaluation of Remote Sensing Image Retrievalcited by 97
  19. 2019SEN12MS -- A Curated Dataset of Georeferenced Multi-Spectral Sentinel-1/2 Imagery for Deep Learning and Data Fusioncited by 95
  20. 2021BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrievalcited by 92
  21. 2022DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentationcited by 78
  22. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 68
  23. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  24. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  25. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  26. 2020MLRSNet: A Multi-label High Spatial Resolution Remote Sensing Dataset for Semantic Scene Understandingcited by 59
  27. 2024Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensingcited by 47
  28. 2024Bridging Remote Sensors with Multisensor Geospatial Foundation Modelscited by 46
  29. 2019In-domain representation learning for remote sensingcited by 41
  30. 2023FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagerycited by 41
  31. 2022Self-supervised Vision Transformers for Joint SAR-optical Representation Learningcited by 40
  32. 2016Fully Convolutional Networks for Dense Semantic Labelling of High-Resolution Aerial Imagerycited by 36
  33. 2020Semi-Supervised Semantic Segmentation in Earth Observation: The MiniFrance Suite, Dataset Analysis and Multi-task Network Studycited by 28
  34. 2020ForestNet: Classifying Drivers of Deforestation in Indonesia using Deep Learning on Satellite Imagerycited by 27
  35. 2022Landslide4Sense: Reference Benchmark Data and Deep Learning Models for Landslide Detectioncited by 26
  36. 2024PhilEO Bench: Evaluating Geo-Spatial Foundation Modelscited by 25
  37. 2022Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domaincited by 22
  38. 2023Mapping smallholder cashew plantations to inform sustainable tree crop expansion in Benincited by 10

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025REMSA: Foundation Model Selection for Remote Sensing via a Constraint-Aware Agentcited by 1
  2. 2026Geometric Consistency Protocol for Foundation Model Features in Multi-View Satellite Imagerycited by 1
  3. 2025EoS-FM: Can an Ensemble of Specialist Models act as a Generalist Feature Extractor?cited by 0