# GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection

arXiv: https://arxiv.org/abs/2603.01161 · HTML: https://mlsanity.com/paper/2603.01161/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 72 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [A Transformer-Based Siamese Network for Change Detection](https://mlsanity.com/paper/2201.01293/) (2022)
- [LocalMamba: Visual State Space Model with Windowed Selective Scan](https://mlsanity.com/paper/2403.09338/) (2024)
- [ChangeMamba: Remote Sensing Change Detection With Spatiotemporal State Space Model](https://mlsanity.com/paper/2404.03425/) (2024)
- [RSMamba: Remote Sensing Image Classification with State Space Model](https://mlsanity.com/paper/2403.19654/) (2024)
- [Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model](https://mlsanity.com/paper/1909.07726/) (2019)
- [CDMamba: Incorporating Local Clues into Mamba for Remote Sensing Image Binary Change Detection](https://mlsanity.com/paper/2406.04207/) (2024)
- [RS3Mamba: Visual State Space Model for Remote Sensing Images Semantic Segmentation](https://mlsanity.com/paper/2404.02457/) (2024)
- [Remote Sensing Change Detection With Transformers Trained from Scratch](https://mlsanity.com/paper/2304.06710/) (2023)
- [Detecting Urban Changes with Recurrent Neural Networks from Multitemporal Sentinel-2 Data](https://mlsanity.com/paper/1910.07778/) (2019)
- [MIScnn: A Framework for Medical Image Segmentation with Convolutional Neural Networks and Deep Learning](https://mlsanity.com/paper/1910.09308/) (2019)
- [CM-UNet: Hybrid CNN-Mamba UNet for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2405.10530/) (2024)
- [SARAS-Net: Scale and Relation Aware Siamese Network for Change Detection](https://mlsanity.com/paper/2212.01287/) (2022)
- [LCD-Net: A Lightweight Remote Sensing Change Detection Network Combining Feature Fusion and Gating Mechanism](https://mlsanity.com/paper/2410.11580/) (2024)
- [Advancing Weakly-Supervised Change Detection in Satellite Images via Adversarial Class Prompting](https://mlsanity.com/paper/2508.17186/) (2025)
- [Operational Neural Networks for Parameter-Efficient Hyperspectral Single-Image Super-Resolution](https://mlsanity.com/paper/2303.16636/) (2023)

## Abstract (arXiv)

Change detection (CD) in remote sensing aims to identify semantic differences between satellite images captured at different times. While deep learning has significantly advanced this field, existing approaches based on convolutional neural networks (CNNs), transformers and Selective State Space Models (SSMs) still struggle to precisely delineate change regions. In particular, traditional transformer-based methods suffer from quadratic computational complexity when applied to very high-resolution (VHR) satellite images and often perform poorly with limited training data, leading to under-utilization of the rich spatial information available in VHR imagery. We present GRAD-Former, a novel framework that enhances contextual understanding while maintaining efficiency through reduced model size. The proposed framework consists of a novel encoder with Adaptive Feature Relevance and Refinement (AFRAR) module, fusion and decoder blocks. AFRAR integrates global-local contextual awareness through two proposed components: the Selective Embedding Amplification (SEA) module and the Global-Local Feature Refinement (GLFR) module. SEA and GLFR leverage gating mechanisms and differential attention, respectively, which generates multiple softmax heaps to capture important features while minimizing the captured irreverent features. Multiple experiments across three challenging CD datasets (LEVIR-CD, CDD, DSIFN-CD) demonstrate GRAD-Former's superior performance compared to existing approaches. Notably, GRAD-Former outperforms the current state-of-the-art models across all the metrics and all the datasets while using fewer parameters. Our framework establishes a new benchmark for remote sensing change detection performance. Our code will be released at: https://github.com/Ujjwal238/GRAD-Former