# Predicting Gradient is Better: Exploring Self-Supervised Learning for SAR ATR with a Joint-Embedding Predictive Architecture

arXiv: https://arxiv.org/abs/2311.15153 · HTML: https://mlsanity.com/paper/2311.15153/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 78 references that are in the map.
- First posted on arXiv 26 Nov 2023.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture](https://mlsanity.com/paper/2301.08243/) (2023)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [Masked Feature Prediction for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2112.09133/) (2021)
- [Scale-MAE: A Scale-Aware Masked Autoencoder for Multiscale Geospatial Representation Learning](https://mlsanity.com/paper/2212.14532/) (2022)
- [Rethinking and Improving Relative Position Encoding for Vision Transformer](https://mlsanity.com/paper/2107.14222/) (2021)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [Towards Large-Scale Small Object Detection: Survey and Benchmarks](https://mlsanity.com/paper/2207.14096/) (2022)
- [Revealing the Dark Secrets of Masked Image Modeling](https://mlsanity.com/paper/2205.13543/) (2022)
- [Pre-training without Natural Images](https://mlsanity.com/paper/2101.08515/) (2021)
- [Large Selective Kernel Network for Remote Sensing Object Detection](https://mlsanity.com/paper/2303.09030/) (2023)
- [BigEarthNet-MM: A Large Scale Multi-Modal Multi-Label Benchmark Archive for Remote Sensing Image Classification and Retrieval](https://mlsanity.com/paper/2105.07921/) (2021)
- [Battle of the Backbones: A Large-Scale Comparison of Pretrained Models across Computer Vision Tasks](https://mlsanity.com/paper/2310.19909/) (2023)
- [Masked Frequency Modeling for Self-Supervised Visual Pre-Training](https://mlsanity.com/paper/2206.07706/) (2022)
- [Informative Dropout for Robust Representation Learning: A Shape-bias Perspective](https://mlsanity.com/paper/2008.04254/) (2020)
- [Hard Patches Mining for Masked Image Modeling](https://mlsanity.com/paper/2304.05919/) (2023)
- [Feature Guided Masked Autoencoder for Self-supervised Learning in Remote Sensing](https://mlsanity.com/paper/2310.18653/) (2023)
- [On Data Scaling in Masked Image Modeling](https://mlsanity.com/paper/2206.04664/) (2022)
- [Masked Image Modeling with Local Multi-Scale Reconstruction](https://mlsanity.com/paper/2303.05251/) (2023)
- [TOV: The Original Vision Model for Optical Remote Sensing Image Understanding via Self-supervised Learning](https://mlsanity.com/paper/2204.04716/) (2022)
- [PixMIM: Rethinking Pixel Reconstruction in Masked Image Modeling](https://mlsanity.com/paper/2303.02416/) (2023)
- [Efficient Self-supervised Vision Pretraining with Local Masked Reconstruction](https://mlsanity.com/paper/2206.00790/) (2022)
- [Automatic Target Recognition on Synthetic Aperture Radar Imagery: A Survey](https://mlsanity.com/paper/2007.02106/) (2020)
- [Physically Explainable CNN for SAR Image Classification](https://mlsanity.com/paper/2110.14144/) (2021)
- [Scattering Model Guided Adversarial Examples for SAR Target Recognition: Attack and Defense](https://mlsanity.com/paper/2209.04779/) (2022)
- [Hierarchical Disentanglement-Alignment Network for Robust SAR Vehicle Recognition](https://mlsanity.com/paper/2304.03550/) (2023)

## Built on by (most cited first, 30 of 46)

- [AnySat: One Earth Observation Model for Many Resolutions, Scales, and Modalities](https://mlsanity.com/paper/2412.14123/) (2024)
- [Towards a Unified Copernicus Foundation Model for Earth Vision](https://mlsanity.com/paper/2503.11849/) (2025)
- [TESSERA: Temporal Embeddings of Surface Spectra for Earth Representation and Analysis](https://mlsanity.com/paper/2506.20380/) (2025)
- [SARATR-X: Toward Building A Foundation Model for SAR Target Recognition](https://mlsanity.com/paper/2405.09365/) (2024)
- [SM3Det: A Unified Model for Multi-Modal Remote Sensing Object Detection](https://mlsanity.com/paper/2412.20665/) (2024)
- [A Survey on Remote Sensing Foundation Models: From Vision to Multimodality](https://mlsanity.com/paper/2503.22081/) (2025)
- [Generic Knowledge Boosted Pre-training For Remote Sensing Images](https://mlsanity.com/paper/2401.04614/) (2024)
- [Strip R-CNN: Large Strip Convolution for Remote Sensing Object Detection](https://mlsanity.com/paper/2501.03775/) (2025)
- [RSAR: Restricted State Angle Resolver and Rotated SAR Benchmark](https://mlsanity.com/paper/2501.04440/) (2025)
- [ATRNet-STAR: A Large Dataset and Benchmark Towards Remote Sensing Object Recognition in the Wild](https://mlsanity.com/paper/2501.13354/) (2025)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Prompting](https://mlsanity.com/paper/2504.12795/) (2025)
- [M4-SAR: A Multi-Resolution, Multi-Polarization, Multi-Scene, Multi-Source Dataset and Benchmark for optical-SAR Object Detection](https://mlsanity.com/paper/2505.10931/) (2025)
- [Visual Instruction Pretraining for Domain-Specific Foundation Models](https://mlsanity.com/paper/2509.17562/) (2025)
- [SARMAE: Masked Autoencoder for SAR Representation Learning](https://mlsanity.com/paper/2512.16635/) (2025)
- [Step-wise Distribution Alignment Guided Style Prompt Tuning for Source-free Cross-domain Few-shot Learning](https://mlsanity.com/paper/2411.10070/) (2024)
- [A Survey on SAR ship classification using Deep Learning](https://mlsanity.com/paper/2503.11906/) (2025)
- [REJEPA: A Novel Joint-Embedding Predictive Architecture for Efficient Remote Sensing Image Retrieval](https://mlsanity.com/paper/2504.03169/) (2025)
- [GDROS: A Geometry-Guided Dense Registration Framework for Optical-SAR Images under Large Geometric Transformations](https://mlsanity.com/paper/2511.00598/) (2025)
- [CrossEarth-SAR: A SAR-Centric and Billion-Scale Geospatial Foundation Model for Domain Generalizable Semantic Segmentation](https://mlsanity.com/paper/2603.12008/) (2026)
- [SAFE: a SAR Feature Extractor based on self-supervised learning and masked Siamese ViTs](https://mlsanity.com/paper/2407.00851/) (2024)
- [On the Status of Foundation Models for SAR Imagery](https://mlsanity.com/paper/2509.21722/) (2025)
- [FUSAR-KLIP: Towards Multimodal Foundation Models for Remote Sensing](https://mlsanity.com/paper/2509.23927/) (2025)
- [MAESTRO: Masked AutoEncoders for Multimodal, Multitemporal, and Multispectral Earth Observation Data](https://mlsanity.com/paper/2508.10894/) (2025)
- [SARFormer -- An Acquisition Parameter Aware Vision Transformer for Synthetic Aperture Radar Data](https://mlsanity.com/paper/2504.08441/) (2025)
- [SARVLM: A Vision Language Foundation Model for Semantic Understanding in SAR Imagery](https://mlsanity.com/paper/2510.22665/) (2025)
- [SpecAware: A Spectral-Content Aware Foundation Model for Unifying Multi-Sensor Learning in Hyperspectral Remote Sensing Mapping](https://mlsanity.com/paper/2510.27219/) (2025)
- [Learning with less: label-efficient land cover classification at very high spatial resolution using self-supervised deep learning](https://mlsanity.com/paper/2511.03004/) (2025)
- [Promptable Foundation Models for SAR Remote Sensing: Adapting the Segment Anything Model for Snow Avalanche Segmentation](https://mlsanity.com/paper/2601.01213/) (2026)
- [Enhanced LULC Segmentation via Lightweight Model Refinements on ALOS-2 SAR Data](https://mlsanity.com/paper/2601.15705/) (2026)

## Abstract (arXiv)

The growing Synthetic Aperture Radar (SAR) data has the potential to build a foundation model through Self-Supervised Learning (SSL) methods, which can achieve various SAR Automatic Target Recognition (ATR) tasks with pre-training in large-scale unlabeled data and fine-tuning in small labeled samples. SSL aims to construct supervision signals directly from the data, which minimizes the need for expensive expert annotation and maximizes the use of the expanding data pool for a foundational model. This study investigates an effective SSL method for SAR ATR, which can pave the way for a foundation model in SAR ATR. The primary obstacles faced in SSL for SAR ATR are the small targets in remote sensing and speckle noise in SAR images, corresponding to the SSL approach and signals. To overcome these challenges, we present a novel Joint-Embedding Predictive Architecture for SAR ATR (SAR-JEPA), which leverages local masked patches to predict the multi-scale SAR gradient representations of unseen context. The key aspect of SAR-JEPA is integrating SAR domain features to ensure high-quality self-supervised signals as target features. Besides, we employ local masks and multi-scale features to accommodate the various small targets in remote sensing. By fine-tuning and evaluating our framework on three target recognition datasets (vehicle, ship, and aircraft) with four other datasets as pre-training, we demonstrate its outperformance over other SSL methods and its effectiveness with increasing SAR data. This study showcases the potential of SSL for SAR target recognition across diverse targets, scenes, and sensors.Our codes and weights are available in urlhttps://github.com/waterdisappear/SAR-JEPA.