mlsanityData to 14 Oct 2026

Developing a foundation model for high-resolution remote sensing data of the Netherlands

arXiv 2605.10184 ↗First posted 11 May 2026cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

We develop a foundation model using 1.2m high resolution satellite images of the Netherlands. By combining a Convolutional Neural Network and a Vision Transformer, the model captures both low- and high-frequency landscape features, such as fine textures, edges, and small objects as well as large terrain structures, elevation patterns, and land-cover distributions. Leveraging temporal data as input, the model learns from broader contextual information across time, allowing the model to exploit the temporal dependencies, such as topographic features, land-cover changes, and seasonal dynamics. These additional constraints reduce feature ambiguity, improve representation learning, and enable better generalization with fewer labeled samples. The foundation model is evaluated on multiple downstream tasks, ranging from use cases within the Netherlands to global benchmarking datasets. On the vegetation monitoring dataset of the Netherlands, the model shows clear performance improvements by incorporating temporal information instead of relying on a single time point. Despite using a smaller model and less pretraining data limited to the Netherlands, it achieves competitive results on global benchmarks when compared to state-of-the-art models. These results demonstrate that the model can learn rich, generalizable representations from limited data, achieving competitive performance on global benchmarks while using a fraction of the parameters of larger state-of-the-art remote sensing models. To maximize reproducibility and reuse, we made the scripts and the model accessible on GitHub.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 11 May 2026.

Builds on

27 of 37 references.

References · 27

  1. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  4. 2021SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformerscited by 2,940
  5. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,496
  6. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,271
  7. 2021Transformers in Vision: A Surveycited by 1,013
  8. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 797
  9. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 408
  10. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 308
  11. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
  12. 2021Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Datacited by 215
15 more
  1. 2022SatlasPretrain: A Large-Scale Dataset for Remote Sensing Image Understandingcited by 167
  2. 2023Foundation Models for Generalist Geospatial Artificial Intelligencecited by 149
  3. 2018Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networkscited by 110
  4. 2023Towards Geospatial Foundation Models via Continual Pretrainingcited by 109
  5. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  6. 2024Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagerycited by 83
  7. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  8. 2021Self-Supervised Material and Texture Representation Learning for Remote Sensing Taskscited by 49
  9. 2023CMID: A Unified Self-Supervised Learning Framework for Remote Sensing Image Understandingcited by 48
  10. 2024reBEN: Refined BigEarthNet Dataset for Remote Sensing Image Analysiscited by 48
  11. 2024Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensingcited by 47
  12. 2021Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Imagescited by 31
  13. 2022Consecutive Pretraining: A Knowledge Transfer Learning Strategy with Relevant Unlabeled Data for Remote Sensing Domaincited by 22
  14. 2022Swin MAE: Masked Autoencoders for Small Datasetscited by 17
  15. 2024RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Modelcited by 12

Built on by

Not cited yet.