# Weakly Supervised Change Detection via Knowledge Distillation and Multiscale Sigmoid Inference

arXiv: https://arxiv.org/abs/2403.05796 · HTML: https://mlsanity.com/paper/2403.05796/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Detection and dense prediction distillation, one of 12 directions in Knowledge distillation.
- Builds on 26 of its 43 references that are in the map.
- First posted on arXiv 9 Mar 2024.

## Placed in

- Knowledge distillation / [Detection and dense prediction distillation](https://mlsanity.com/distill/detection-and-dense-prediction-distillation/)

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Born Again Neural Networks](https://mlsanity.com/paper/1805.04770/) (2018)
- [Knowledge Distillation and Student-Teacher Learning for Visual Intelligence: A Review and New Outlooks](https://mlsanity.com/paper/2004.05937/) (2020)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [Self-supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2004.04581/) (2020)
- [Structured Knowledge Distillation for Dense Prediction](https://mlsanity.com/paper/1903.04197/) (2019)
- [Object Region Mining with Adversarial Erasing: A Simple Classification to Semantic Segmentation Approach](https://mlsanity.com/paper/1703.08448/) (2017)
- [Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentation](https://mlsanity.com/paper/1603.06098/) (2016)
- [Weakly-Supervised Semantic Segmentation via Sub-category Exploration](https://mlsanity.com/paper/2008.01183/) (2020)
- [Knowledge Adaptation for Efficient Semantic Segmentation](https://mlsanity.com/paper/1903.04688/) (2019)
- [Building Change Detection for Remote Sensing Images Using a Dual Task Constrained Deep Siamese Convolutional Network Model](https://mlsanity.com/paper/1909.07726/) (2019)
- [Unpaired Multi-modal Segmentation via Knowledge Distillation](https://mlsanity.com/paper/2001.03111/) (2020)
- [Reducing Information Bottleneck for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2110.06530/) (2021)
- [Fully Convolutional Change Detection Framework with Generative Adversarial Network for Unsupervised, Weakly Supervised and Regional Supervised Change Detection](https://mlsanity.com/paper/2201.06030/) (2022)
- [Efficient Medical Image Segmentation Based on Knowledge Distillation](https://mlsanity.com/paper/2108.09987/) (2021)
- [Structural and Statistical Texture Knowledge Distillation for Semantic Segmentation](https://mlsanity.com/paper/2305.03944/) (2023)
- [Deep Learning for Change Detection in Remote Sensing Images: Comprehensive Review and Meta-Analysis](https://mlsanity.com/paper/2006.05612/) (2020)
- [Knowledge Distillation in Deep Learning and its Applications](https://mlsanity.com/paper/2007.09029/) (2020)
- [A Weakly Supervised Convolutional Network for Change Segmentation and Classification](https://mlsanity.com/paper/2011.03577/) (2020)
- [Background-Mixed Augmentation for Weakly Supervised Change Detection](https://mlsanity.com/paper/2211.11478/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Survey of Sample-Efficient Deep Learning for Change Detection in Remote Sensing: Tasks, Strategies, and Challenges](https://mlsanity.com/paper/2502.02835/) (2025)
- [Applications of Knowledge Distillation in Remote Sensing: A Survey](https://mlsanity.com/paper/2409.12111/) (2024)

## Abstract (arXiv)

Change detection, which aims to detect spatial changes from a pair of multi-temporal images due to natural or man-made causes, has been widely applied in remote sensing, disaster management, urban management, etc. Most existing change detection approaches, however, are fully supervised and require labor-intensive pixel-level labels. To address this, we develop a novel weakly supervised change detection technique via Knowledge Distillation and Multiscale Sigmoid Inference (KD-MSI) that leverages image-level labels. In our approach, the Class Activation Maps (CAM) are utilized not only to derive a change probability map but also to serve as a foundation for the knowledge distillation process. This is done through a joint training strategy of the teacher and student networks, enabling the student network to highlight potential change areas more accurately than teacher network based on image-level labels. Moreover, we designed a Multiscale Sigmoid Inference (MSI) module as a post processing step to further refine the change probability map from the trained student network. Empirical results on three public datasets, i.e., WHU-CD, DSIFN-CD, and LEVIR-CD, demonstrate that our proposed technique, with its integrated training strategy, significantly outperforms the state-of-the-art.