mlsanityData to 7 Oct 2026

Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing

arXiv 2310.18653 ↗First posted 28 Oct 2023cs.CVcited by 48

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning guided by masked image modelling, such as Masked AutoEncoder (MAE), has attracted wide attention for pretraining vision transformers in remote sensing. However, MAE tends to excessively focus on pixel details, thereby limiting the model's capacity for semantic understanding, in particular for noisy SAR images. In this paper, we explore spectral and spatial remote sensing image features as improved MAE-reconstruction targets. We first conduct a study on reconstructing various image features, all performing comparably well or better than raw pixels. Based on such observations, we propose Feature Guided Masked Autoencoder (FG-MAE): reconstructing a combination of Histograms of Oriented Graidents (HOG) and Normalized Difference Indices (NDI) for multispectral images, and reconstructing HOG for SAR images. Experimental results on three downstream tasks illustrate the effectiveness of FG-MAE with a particular boost for SAR imagery. Furthermore, we demonstrate the well-inherited scalability of FG-MAE and release a first series of pretrained vision transformers for medium resolution SAR and multispectral images.

Key facts. Cited by 48 papers in the map (data through 7 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 28 Oct 2023.

Builds on

18 of 39 references.

References · 18

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  5. 2017EuroSAT: A Novel Dataset and Deep Learning Benchmark for Land Use and Land Cover Classificationcited by 1,488
  6. 2018Datasheets for Datasetscited by 1,371
  7. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  8. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  9. 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
  10. 2022SatMAE: Pre-training Transformers for Temporal and Multi-Spectral Satellite Imagerycited by 402
  11. 2022Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learningcited by 302
  12. 2019Kornia: an Open Source Differentiable Computer Vision Library for PyTorchcited by 231
6 more
  1. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  2. 2018Tile2Vec: Unsupervised representation learning for spatially distributed datacited by 109
  3. 2021BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrievalcited by 91
  4. 2022SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observationcited by 82
  5. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  6. 2022EarthNets: Empowering AI in Earth Observationcited by 40

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 14; 2025: 21; 2026: 12.

48 papers.

Sort:

By area:

  1. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 97
  2. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 72
  3. 2024MTP: Advancing Remote Sensing Foundation Model via Multi-Task Pretrainingcited by 66
  4. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  5. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  6. 2024Cross-Scale MAE: A Tale of Multi-Scale Exploitation in Remote Sensingcited by 47
  7. 2024SARDet-100K: Towards Open-Source Benchmark and ToolKit for Large-Scale SAR Object Detectioncited by 47
  8. 2023Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecturecited by 43
  9. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  10. 2025Panopticon: Advancing Any-Sensor Foundation Models for Earth Observationcited by 34
  11. 2024Multi-Label Guided Soft Contrastive Learning for Efficient Earth Observation Pretrainingcited by 27
  12. 2024SARATR-X: Toward Building A Foundation Model for SAR Target Recognitioncited by 25
36 more
  1. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  2. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  3. 2024Generic Knowledge Boosted Pre-training For Remote Sensing Imagescited by 16
  4. 2025RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretationcited by 16
  5. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  6. 2024SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensingcited by 10
  7. 2025Geospatial Foundation Models to Enable Progress on Sustainable Development Goalscited by 9
  8. 2024DisCoM-KD: Cross-Modal Knowledge Distillation via Disentanglement Representation and Adversarial Learningcited by 6
  9. 2025DOFA-CLIP: Multimodal Vision-Language Foundation Models for Earth Observationcited by 6
  10. 2024Adaptive Self-Supervised Learning Strategies for Dynamic On-Device LLM Personalizationcited by 5
  11. 2025Ice-FMBench: A Foundation Model Benchmark for Sea Ice Type Segmentationcited by 5
  12. 2025A Complex-valued SAR Foundation Model Based on Physically Inspired Representation Learningcited by 4
  13. 2025A Genealogy of Foundation Models in Remote Sensingcited by 4
  14. 2025GeoCrossBench: Cross-Band Generalization for Remote Sensingcited by 4
  15. 2025Unsupervised Time-Series Signal Analysis with Autoencoders and Vision Transformers: A Review of Architectures and Applicationscited by 3
  16. 2025Landsat-Bench: Datasets and Benchmarks for Landsat Foundation Modelscited by 3
  17. 2025MSSDF: Modality-Shared Self-supervised Distillation for High-Resolution Multi-modal Remote Sensing Image Learningcited by 2
  18. 2025MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Datacited by 2
  19. 2026HighFM: Towards a Foundation Model for Learning Representations from High-Frequency Earth Observation Datacited by 2
  20. 2025Learning with less: label-efficient land cover classification at very high spatial resolution using self-supervised deep learningcited by 1
  21. 2026SIGMAE: A Spectral-Index-Guided Foundation Model for Multispectral Remote Sensingcited by 1
  22. 2026How to Embed Matters: Evaluation of EO Embedding Design Choicescited by 1
  23. 2026SLED: Scalable Location Encoding via Distillationcited by 1
  24. 2024Semi Supervised Heterogeneous Domain Adaptation via Disentanglement and Pseudo-Labellingcited by 0
  25. 2024Multispectral Texture Synthesis using RGB Convolutional Neural Networkscited by 0
  26. 2025Enhancing SAR Object Detection with Self-Supervised Pre-training on Masked Auto-Encoderscited by 0
  27. 2025Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sourcescited by 0
  28. 2025DI3CL: Contrastive Learning With Dynamic Instances and Contour Consistency for SAR Land-Cover Classification Foundation Modelcited by 0
  29. 2025CrossEarth-Gate: Fisher-Guided Adaptive Tuning Engine for Efficient Adaptation of Cross-Domain Remote Sensing Semantic Segmentationcited by 0
  30. 2026OceanMAE: A Foundation Model for Ocean Remote Sensingcited by 0
  31. 2026Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencodercited by 0
  32. 2026GNSS-FM: A Self-Supervised Foundation Model for Daily GNSS Displacement Time Seriescited by 0
  33. 2026SAMBA: A Scatter-Guided Masked Bidirectional Mamba Foundation Model for SAR Target Recognitioncited by 0
  34. 2026PhenoEmbed: Self-Supervised Multispectral UAV Time-Series Embeddings for Individual Tree Crown Phenologycited by 0
  35. 2026Global-Scale Self-Supervised Spatiotemporal Learning for NDVI Time-Series Reconstructioncited by 0
  36. 2026Multimodal-Multiresolution Foundation Model for Lunar Remote Sensingcited by 0