mlsanityData to 7 Oct 2026

Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning

arXiv 2212.14532 ↗First posted 30 Dec 2022cs.CVcited by 302

Not on the map. Why?

Abstract (arXiv)

Large, pretrained models are commonly finetuned with imagery that is heavily augmented to mimic different conditions and scales, with the resulting models used for various tasks with imagery from a range of spatial scales. Such models overlook scale-specific information in the data for scale-dependent domains, such as remote sensing. In this paper, we present Scale-MAE, a pretraining method that explicitly learns relationships between data at different, known scales throughout the pretraining process. Scale-MAE pretrains a network by masking an input image at a known input scale, where the area of the Earth covered by the image determines the scale of the ViT positional encoding, not the image resolution. Scale-MAE encodes the masked image with a standard ViT backbone, and then decodes the masked image through a bandpass filter to reconstruct low/high frequency images at lower/higher scales. We find that tasking the network with reconstructing both low/high frequency images leads to robust multiscale representations for remote sensing imagery. Scale-MAE achieves an average of a 2.4 - 5.6% non-parametric kNN classification improvement across eight remote sensing datasets compared to current state-of-the-art and obtains a 0.9 mIoU to 1.7 mIoU improvement on the SpaceNet building segmentation transfer task for a range of evaluation scales.

Key facts. Cited by 302 papers in the map (data through 7 Oct 2026). Builds on 38 of its 68 references that are in the map. First posted on arXiv 30 Dec 2022.

Builds on

38 of 68 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  5. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  6. 2016Feature Pyramid Networks for Object Detectioncited by 5,021
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2015SSD: Single Shot MultiBox Detectorcited by 3,125
  9. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  12. 2018Unsupervised Feature Learning via Non-Parametric Instance-level Discriminationcited by 1,838
26 more
  1. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  2. 2021Multiscale Vision Transformerscited by 896
  3. 2017Remote Sensing Image Scene Classification: Benchmark and State of the Artcited by 791
  4. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  5. 2020Learning Continuous Image Representation with Local Implicit Image Functioncited by 550
  6. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  7. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  8. 2017Functional Map of the Worldcited by 338
  9. 2022Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-trainingcited by 263
  10. 2020Geography-Aware Self-Supervised Learningcited by 190
  11. 2019Meta-SR: A Magnification-Arbitrary Network for Super-Resolutioncited by 180
  12. 2018SpaceNet: A Remote Sensing Dataset and Challenge Seriescited by 178
  13. 2018Deep Learning for Single Image Super-Resolution: A Brief Reviewcited by 158
  14. 2018Land-Cover Classification with High-Resolution Remote Sensing Images Using Transferable Deep Modelscited by 118
  15. 2022ConvMAE: Masked Convolution Meets Masked Autoencoderscited by 105
  16. 2019A Deep Journey into Super-resolution: A surveycited by 86
  17. 2020MLRSNet: A Multi-label High Spatial Resolution Remote Sensing Dataset for Semantic Scene Understandingcited by 59
  18. 2018The Effects of Super-Resolution on Object Detection Performance in Satellite Imagerycited by 57
  19. 2021UltraSR: Spatial Encoding is a Missing Key for Implicit Image Function-based Arbitrary-Scale Super-Resolutioncited by 56
  20. 2018A Fully Progressive Approach to Single-Image Super-Resolutioncited by 46
  21. 2019In-domain representation learning for remote sensingcited by 41
  22. 2022EarthNets: Empowering AI in Earth Observationcited by 40
  23. 2022xView3-SAR: Detecting Dark Fishing Activity Using Synthetic Aperture Radar Imagerycited by 25
  24. 2021Spatial-Temporal Super-Resolution of Satellite Imagery via Conditional Pixel Synthesiscited by 20
  25. 2020AiRound and CV-BrCT: Novel Multi-View Datasets for Scene Classificationcited by 10
  26. 2022Snowpack Estimation in Key Mountainous Water Basins from Openly-Available, Multimodal Data Sourcescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 29; 2024: 67; 2025: 109; 2026: 97.

302 papers.

Sort:

By area:

  1. 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensingcited by 357
  2. 2023ClimaX: A foundation model for weather and climatecited by 325
  3. 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 202
  4. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 181
  5. 2023RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Modelcited by 180
  6. 2023CROMA: Remote Sensing Representations with Contrastive Radar-Optical Masked Autoencoderscited by 167
  7. 2024Prithvi-EO-2.0: A Versatile Multi-Temporal Foundation Model for Earth Observation Applicationscited by 139
  8. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 133
  9. 2023SkyScript: A Large and Semantically Diverse Vision-Language Dataset for Remote Sensingcited by 123
  10. 2023Lightweight, Pre-trained Transformers for Remote Sensing Timeseriescited by 102
  11. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 97
  12. 2024Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagerycited by 82
38 more
  1. 2024Mission Critical -- Satellite Data is a Distinct Modality in Machine Learningcited by 76
  2. 2024MMEarth: Exploring Multi-Modal Pretext Tasks For Geospatial Representation Learningcited by 76
  3. 2024AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalitiescited by 73
  4. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 72
  5. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  6. 2024OmniSat: Self-Supervised Modality Fusion for Earth Observationcited by 63
  7. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  8. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  9. 2023Very high resolution canopy height maps from RGB imagery using self-supervised vision transformer and convolutional decoder trained on Aerial Lidarcited by 60
  10. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  11. 2024Locate Anything on Earth: Advancing Open-Vocabulary Object Detection for Remote Sensing Communitycited by 50
  12. 2023SSL4EO-L: Datasets and Foundation Models for Landsat Imagerycited by 49
  13. 2023Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensingcited by 48
  14. 2024Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensingcited by 47
  15. 2023Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecturecited by 43
  16. 2024SpectralEarth: Training Hyperspectral Foundation Models at Scalecited by 42
  17. 2024MetaEarth: A Generative Foundation Model for Global-Scale Remote Sensing Image Generationcited by 41
  18. 2025Galileo: Learning Global & Local Features of Many Remote Sensing Modalitiescited by 41
  19. 2025RemoteSAM: Towards Segment Anything for Earth Observationcited by 38
  20. 2024SwitchTab: Switched Autoencoders Are Effective Tabular Learnerscited by 37
  21. 2024Changen2: Multi-Temporal Remote Sensing Generative Change Foundation Modelcited by 37
  22. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  23. 2025Panopticon: Advancing Any-Sensor Foundation Models for Earth Observationcited by 34
  24. 2024Deep Learning for Cross-Domain Data Fusion in Urban Computing: Taxonomy, Advances, and Outlookcited by 33
  25. 2024TaxaBind: A Unified Embedding Space for Ecological Applicationscited by 33
  26. 2024GeoSynth: Contextually-Aware High-Resolution Satellite Image Synthesiscited by 32
  27. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  28. 2023FoMo: Multi-Modal, Multi-Scale and Multi-Task Remote Sensing Foundation Models for Forest Monitoringcited by 30
  29. 2023USat: A Unified Self-Supervised Encoder for Multi-Sensor Satellite Imagerycited by 28
  30. 2024Masked Image Modeling: A Surveycited by 28
  31. 2023Revisiting pre-trained remote sensing model benchmarks: resizing and normalization matterscited by 27
  32. 2024Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretrainingcited by 27
  33. 2024CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentationcited by 27
  34. 2025SkySense V2: A Unified Foundation Model for Multi-modal Remote Sensingcited by 26
  35. 2024PhilEO Bench: Evaluating Geo-Spatial Foundation Modelscited by 25
  36. 2024SARATR-X: Toward Building A Foundation Model for SAR Target Recognitioncited by 25
  37. 2025TerraFM: A Scalable Foundation Model for Unified Multisensor Earth Observationcited by 25
  38. 2023ReConTab: Regularized Contrastive Representation Learning for Tabular Datacited by 24

50 most cited of 302 All 302 (JSON)