# Remote Sensing Change Detection With Transformers Trained from Scratch

arXiv: https://arxiv.org/abs/2304.06710 · HTML: https://mlsanity.com/paper/2304.06710/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 55 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [A Transformer-Based Siamese Network for Change Detection](https://mlsanity.com/paper/2201.01293/) (2022)
- [Urban Change Detection for Multispectral Earth Observation Using Convolutional Neural Networks](https://mlsanity.com/paper/1810.08468/) (2018)
- [Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model](https://mlsanity.com/paper/1909.07726/) (2019)
- [Changer: Feature Interaction is What You Need for Change Detection](https://mlsanity.com/paper/2209.08290/) (2022)
- [DASNet: Dual attentive fully convolutional siamese networks for change detection of high resolution satellite images](https://mlsanity.com/paper/2003.03608/) (2020)
- [Transformers in Remote Sensing: A Survey](https://mlsanity.com/paper/2209.01206/) (2022)
- [Learning to Measure Change: Fully Convolutional Siamese Metric Networks for Scene Change Detection](https://mlsanity.com/paper/1810.09111/) (2018)
- [Super-resolution-based Change Detection Network with Stacked Attention Module for Images with Different Resolutions](https://mlsanity.com/paper/2103.00188/) (2021)
- [Fully Transformer Network for Change Detection of Remote Sensing Images](https://mlsanity.com/paper/2210.00757/) (2022)

## Built on by (most cited first, 22 of 22)

- [Rethinking Transformers Pre-training for Multi-Spectral Satellite Imagery](https://mlsanity.com/paper/2403.05419/) (2024)
- [Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysis](https://mlsanity.com/paper/2403.19646/) (2024)
- [ELGC-Net: Efficient Local-Global Context Aggregation for Remote Sensing Change Detection](https://mlsanity.com/paper/2403.17909/) (2024)
- [A Survey of Sample-Efficient Deep Learning for Change Detection in Remote Sensing: Tasks, Strategies, and Challenges](https://mlsanity.com/paper/2502.02835/) (2025)
- [Be the Change You Want to See: Revisiting Remote Sensing Change Detection Practices](https://mlsanity.com/paper/2507.03367/) (2025)
- [Continuous Urban Change Detection from Satellite Image Time Series with Temporal Feature Refinement and Multi-Task Integration](https://mlsanity.com/paper/2406.17458/) (2024)
- [PeftCD: Leveraging Vision Foundation Models with Parameter-Efficient Fine-Tuning for Remote Sensing Change Detection](https://mlsanity.com/paper/2509.09572/) (2025)
- [ChangeBind: A Hybrid Change Encoder for Remote Sensing Change Detection](https://mlsanity.com/paper/2404.17565/) (2024)
- [MONITRS: Multimodal Observations of Natural Incidents Through Remote Sensing](https://mlsanity.com/paper/2507.16228/) (2025)
- [SCL: Towards Domain Generalization via Single-Temporal Multimodal Contrastive Learning for Remote Sensing Change Detection](https://mlsanity.com/paper/2404.11326/) (2024)
- [An Effective Weight Initialization Method for Deep Learning: Application to Satellite Image Classification](https://mlsanity.com/paper/2406.00348/) (2024)
- [HyRet-Change: A hybrid retentive network for remote sensing change detection](https://mlsanity.com/paper/2506.12836/) (2025)
- [UAV as Urban Construction Change Monitor: A New Benchmark and Change Captioning Model](https://mlsanity.com/paper/2605.04409/) (2026)
- [Leveraging Fine-Grained Information and Noise Decoupling for Remote Sensing Change Detection](https://mlsanity.com/paper/2404.11318/) (2024)
- [A Remote Sensing Image Change Detection Method Integrating Layer Exchange and Channel-Spatial Differences](https://mlsanity.com/paper/2501.10905/) (2025)
- [Exchange Is All You Need for Remote Sensing Change Detection](https://mlsanity.com/paper/2601.07805/) (2026)
- [HA2F: Dual-module Collaboration-Guided Hierarchical Adaptive Aggregation Framework for Remote Sensing Change Detection](https://mlsanity.com/paper/2601.16573/) (2026)
- [EoCD: Encoder only Remote Sensing Change Detection](https://mlsanity.com/paper/2602.05882/) (2026)
- [Improved MambdaBDA Framework for Robust Building Damage Assessment Across Disaster Domains](https://mlsanity.com/paper/2603.01116/) (2026)
- [GRAD-Former: Gated Robust Attention-based Differential Transformer for Change Detection](https://mlsanity.com/paper/2603.01161/) (2026)
- [Building Change Detection in Earthquake: A Multi-Scale Interaction Network and A Change Detection Dataset](https://mlsanity.com/paper/2606.10329/) (2026)
- [JL1-CC&QA: Extending the JL1-CD Benchmark with Change Captioning and Question Answering](https://mlsanity.com/paper/2606.31745/) (2026)

## Abstract (arXiv)

Current transformer-based change detection (CD) approaches either employ a pre-trained model trained on large-scale image classification ImageNet dataset or rely on first pre-training on another CD dataset and then fine-tuning on the target benchmark. This current strategy is driven by the fact that transformers typically require a large amount of training data to learn inductive biases, which is insufficient in standard CD datasets due to their small size. We develop an end-to-end CD approach with transformers that is trained from scratch and yet achieves state-of-the-art performance on four public benchmarks. Instead of using conventional self-attention that struggles to capture inductive biases when trained from scratch, our architecture utilizes a shuffled sparse-attention operation that focuses on selected sparse informative regions to capture the inherent characteristics of the CD data. Moreover, we introduce a change-enhanced feature fusion (CEFF) module to fuse the features from input image pairs by performing a per-channel re-weighting. Our CEFF module aids in enhancing the relevant semantic changes while suppressing the noisy ones. Extensive experiments on four CD datasets reveal the merits of the proposed contributions, achieving gains as high as 14.27% in intersection-over-union (IoU) score, compared to the best-published results in the literature. Code is available at urlhttps://github.com/mustansarfiaz/ScratchFormer.