mlsanityData to 7 Oct 2026

Self-supervised Vision Transformers for Joint SAR-optical Representation Learning

arXiv 2204.05381 ↗First posted 11 Apr 2022cs.CVcited by 40

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) has attracted much interest in remote sensing and earth observation due to its ability to learn task-agnostic representations without human annotation. While most of the existing SSL works in remote sensing utilize ConvNet backbones and focus on a single modality, we explore the potential of vision transformers (ViTs) for joint SAR-optical representation learning. Based on DINO, a state-of-the-art SSL algorithm that distills knowledge from two augmented views of an input image, we combine SAR and optical imagery by concatenating all channels to a unified input. Subsequently, we randomly mask out channels of one modality as a data augmentation strategy. While training, the model gets fed optical-only, SAR-only, and SAR-optical image pairs learning both inner- and intra-modality representations. Experimental results employing the BigEarthNet-MM dataset demonstrate the benefits of both, the ViT backbones and the proposed multimodal SSL algorithm DINO-MM.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 15 of its 18 references that are in the map. First posted on arXiv 11 Apr 2022.

Builds on

15 of 18 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  8. 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
  9. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  10. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  11. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
  12. 2020Geography-Aware Self-Supervised Learningcited by 190
3 more
  1. 2018Tile2Vec: Unsupervised representation learning for spatially distributed datacited by 109
  2. 2021BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrievalcited by 91
  3. 2021Contrastive Multiview Coding with Electro-optics for SAR Semantic Segmentationcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 6; 2024: 7; 2025: 19; 2026: 4.

40 papers.

Sort:

By area:

  1. 2022Self-supervised Learning in Remote Sensing: A Reviewcited by 137
  2. 2022SSL4EO-S12: A Large-Scale Multi-Modal, Multi-Temporal Dataset for Self-Supervised Learning in Earth Observationcited by 82
  3. 2023A Billion-scale Foundation Model for Remote Sensing Imagescited by 62
  4. 2024PANGAEA: A Global and Inclusive Benchmark for Geospatial Foundation Modelscited by 62
  5. 2025Towards a Unified Copernicus Foundation Model for Earth Visioncited by 52
  6. 2023Decoupling Common and Unique Representations for Multimodal Self-supervised Learningcited by 36
  7. 2025TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysiscited by 35
  8. 2025Panopticon: Advancing Any-Sensor Foundation Models for Earth Observationcited by 34
  9. 2024On the Foundations of Earth and Climate Foundation Modelscited by 31
  10. 2022Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Workscited by 24
  11. 2025FlexiMo: A Flexible Remote Sensing Foundation Modelcited by 24
  12. 2023Extending global-local view alignment for self-supervised learning with remote sensing imagerycited by 20
28 more
  1. 2025A Survey on Remote Sensing Foundation Models: From Vision to Multimodalitycited by 18
  2. 2025RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretationcited by 16
  3. 2026Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodalitycited by 12
  4. 2024SeaMo: A Season-Aware Multimodal Foundation Model for Remote Sensingcited by 10
  5. 2025Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflectioncited by 10
  6. 2024A Survey of the Self Supervised Learning Mechanisms for Vision Transformerscited by 8
  7. 2022Transfer Learning with Pretrained Remote Sensing Transformerscited by 6
  8. 2024Fus-MAE: A cross-attention-based data fusion approach for Masked Autoencoders in remote sensingcited by 6
  9. 2024Increasing the Robustness of Model Predictions to Missing Sensors in Earth Observationcited by 5
  10. 2025Ice-FMBench: A Foundation Model Benchmark for Sea Ice Type Segmentationcited by 5
  11. 2023Transformer-based Multi-Modal Learning for Multi Label Remote Sensing Image Classificationcited by 4
  12. 2025A Complex-valued SAR Foundation Model Based on Physically Inspired Representation Learningcited by 4
  13. 2025A Genealogy of Foundation Models in Remote Sensingcited by 4
  14. 2025Towards Efficient Benchmarking of Foundation Models in Remote Sensing: A Capabilities Encoding Approachcited by 3
  15. 2025Survey of Multimodal Geospatial Foundation Models: Techniques, Applications, and Challengescited by 3
  16. 2026TESSERA v2: Scaling Pixel-wise Earth Foundation Modelscited by 3
  17. 2023Self-Supervised Learning for Improved Synthetic Aperture Sonar Target Recognitioncited by 2
  18. 2025Missing Data as Augmentation in the Earth Observation Domain: A Multi-View Learning Approachcited by 2
  19. 2025MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Datacited by 2
  20. 2023Joint multi-modal Self-Supervised pre-training in Remote Sensing: Application to Methane Source Classificationcited by 1
  21. 2024Estimating Physical Information Consistency of Channel Data Augmentation for Remote Sensing Imagescited by 1
  22. 2025SARFormer -- An Acquisition Parameter Aware Vision Transformer for Synthetic Aperture Radar Datacited by 1
  23. 2025Pan-Arctic Permafrost Landform and Human-built Infrastructure Feature Detection with Vision Transformers and Location Embeddingscited by 1
  24. 2025SSL4SAR: Self-Supervised Learning for Glacier Calving Front Extraction from SAR Imagerycited by 1
  25. 2025SatDINO: A Deep Dive into Self-Supervised Pretraining for Remote Sensingcited by 0
  26. 2025Mapping Hidden Heritage: Self-supervised Pre-training on High-Resolution LiDAR DEM Derivatives for Archaeological Stone Wall Detectioncited by 0
  27. 2026Parameter-Efficient Modality-Balanced Symmetric Fusion for Multimodal Remote Sensing Semantic Segmentationcited by 0
  28. 2026Better with Less: Tackling Heterogeneous Multi-Modal Image Joint Pretraining via Conditioned and Degraded Masked Autoencodercited by 0