# Continual Barlow Twins: continual self-supervised learning for remote sensing semantic segmentation

arXiv: https://arxiv.org/abs/2205.11319 · HTML: https://mlsanity.com/paper/2205.11319/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 75 references that are in the map.
- First posted on arXiv 23 May 2022.

## Builds on (most cited first, 30 of 33)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [Continual Lifelong Learning with Neural Networks: A Review](https://mlsanity.com/paper/1802.07569/) (2018)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Self-supervised Learning: Generative or Contrastive](https://mlsanity.com/paper/2006.08218/) (2020)
- [A Survey on Contrastive Self-supervised Learning](https://mlsanity.com/paper/2011.00362/) (2020)
- [Graph Self-Supervised Learning: A Survey](https://mlsanity.com/paper/2103.00111/) (2021)
- [BigEarthNet: A Large-Scale Benchmark Archive For Remote Sensing Image Understanding](https://mlsanity.com/paper/1902.06148/) (2019)
- [Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data](https://mlsanity.com/paper/2103.16607/) (2021)
- [Geography-Aware Self-Supervised Learning](https://mlsanity.com/paper/2011.09980/) (2020)
- [Self-Supervised Models are Continual Learners](https://mlsanity.com/paper/2112.04215/) (2021)
- [Self-Supervised Representation Learning: Introduction, Advances and Challenges](https://mlsanity.com/paper/2110.09327/) (2021)
- [Self-Supervised Pretraining Improves Self-Supervised Pretraining](https://mlsanity.com/paper/2103.12718/) (2021)
- [DynamicEarthNet: Daily Multi-Spectral Satellite Dataset for Semantic Change Segmentation](https://mlsanity.com/paper/2203.12560/) (2022)
- [Recalibrating Fully Convolutional Networks with Spatial and Channel 'Squeeze & Excitation' Blocks](https://mlsanity.com/paper/1808.08127/) (2018)
- [Remote Sensing Image Scene Classification Meets Deep Learning: Challenges, Methods, Benchmarks, and Opportunities](https://mlsanity.com/paper/2005.01094/) (2020)
- [OBoW: Online Bag-of-Visual-Words Generation for Self-Supervised Learning](https://mlsanity.com/paper/2012.11552/) (2020)
- [Multi-Attention-Network for Semantic Segmentation of Fine Resolution Remote Sensing Images](https://mlsanity.com/paper/2009.02130/) (2020)
- [Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding](https://mlsanity.com/paper/2104.07070/) (2021)
- [Semantic Stereo for Incidental Satellite Images](https://mlsanity.com/paper/1811.08739/) (2018)
- [Geographical Knowledge-driven Representation Learning for Remote Sensing Images](https://mlsanity.com/paper/2107.05276/) (2021)
- [Remote Sensing Image Scene Classification with Self-Supervised Paradigm under Limited Labeled Samples](https://mlsanity.com/paper/2010.00882/) (2020)
- [The Challenges of Continuous Self-Supervised Learning](https://mlsanity.com/paper/2203.12710/) (2022)
- [Incremental Learning for Semantic Segmentation of Large-Scale Remote Sensing Data](https://mlsanity.com/paper/1810.12448/) (2018)
- [The color out of space: learning self-supervised representations for Earth Observation imagery](https://mlsanity.com/paper/2006.12119/) (2020)
- [Global and Local Contrastive Self-Supervised Learning for Semantic Segmentation of HR Remote Sensing Images](https://mlsanity.com/paper/2106.10605/) (2021)

## Built on by (most cited first, 22 of 22)

- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [Self-supervised Learning in Remote Sensing: A Review](https://mlsanity.com/paper/2206.13188/) (2022)
- [SynDrone -- Multi-modal UAV Dataset for Urban Scenarios](https://mlsanity.com/paper/2308.10491/) (2023)
- [Self-supervised remote sensing feature learning: Learning Paradigms, Challenges, and Future Works](https://mlsanity.com/paper/2211.08129/) (2022)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Class Incremental Learning with Self-Supervised Pre-Training and Prototype Learning](https://mlsanity.com/paper/2308.02346/) (2023)
- [Knowledge Accumulation in Continually Learned Representations and the Issue of Feature Forgetting](https://mlsanity.com/paper/2304.00933/) (2023)
- [GeoMultiTaskNet: remote sensing unsupervised domain adaptation using geographical coordinates](https://mlsanity.com/paper/2304.07750/) (2023)
- [Parameter-Efficient Adaptation of Geospatial Foundation Models through Embedding Deflection](https://mlsanity.com/paper/2503.09493/) (2025)
- [Continual Learning with Neuromorphic Computing: Foundations, Methods, and Emerging Applications](https://mlsanity.com/paper/2410.09218/) (2024)
- [From MNIST to ImageNet and Back: Benchmarking Continual Curriculum Learning](https://mlsanity.com/paper/2303.11076/) (2023)
- [Branch-Tuning: Balancing Stability and Plasticity for Continual Self-Supervised Learning](https://mlsanity.com/paper/2403.18266/) (2024)
- [Continual Panoptic Perception: Towards Multi-modal Incremental Interpretation of Remote Sensing Images](https://mlsanity.com/paper/2407.14242/) (2024)
- [Out-of-distribution forgetting: vulnerability of continual learning to intra-class distribution shift](https://mlsanity.com/paper/2306.00427/) (2023)
- [Cross-sensor self-supervised training and alignment for remote sensing](https://mlsanity.com/paper/2405.09922/) (2024)
- [Towards Non-Exemplar Semi-Supervised Class-Incremental Learning](https://mlsanity.com/paper/2403.18291/) (2024)
- [Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation](https://mlsanity.com/paper/2406.18037/) (2024)
- [Continual Self-Supervised Learning with Masked Autoencoders in Remote Sensing](https://mlsanity.com/paper/2506.21312/) (2025)
- [A Plasticity-Aware Method for Continual Self-Supervised Learning in Remote Sensing](https://mlsanity.com/paper/2503.24088/) (2025)
- [CL-BioGAN: Biologically-Inspired Cross-Domain Continual Learning for Hyperspectral Anomaly Detection](https://mlsanity.com/paper/2505.11796/) (2025)
- [Evolving Without Ending: Unifying Multimodal Incremental Learning for Continual Panoptic Perception](https://mlsanity.com/paper/2601.15643/) (2026)
- [Lifelong Representations: A Survey on Continual Self-Supervised Learning for Vision Models](https://mlsanity.com/paper/2607.09785/) (2026)

## Abstract (arXiv)

In the field of Earth Observation (EO), Continual Learning (CL) algorithms have been proposed to deal with large datasets by decomposing them into several subsets and processing them incrementally. The majority of these algorithms assume that data is (a) coming from a single source, and (b) fully labeled. Real-world EO datasets are instead characterized by a large heterogeneity (e.g., coming from aerial, satellite, or drone scenarios), and for the most part they are unlabeled, meaning they can be fully exploited only through the emerging Self-Supervised Learning (SSL) paradigm. For these reasons, in this paper we propose a new algorithm for merging SSL and CL for remote sensing applications, that we call Continual Barlow Twins (CBT). It combines the advantages of one of the simplest self-supervision techniques, i.e., Barlow Twins, with the Elastic Weight Consolidation method to avoid catastrophic forgetting. In addition, for the first time we evaluate SSL methods on a highly heterogeneous EO dataset, showing the effectiveness of these strategies on a novel combination of three almost non-overlapping domains datasets (airborne Potsdam dataset, satellite US3D dataset, and drone UAVid dataset), on a crucial downstream task in EO, i.e., semantic segmentation. Encouraging results show the superiority of SSL in this setting, and the effectiveness of creating an incremental effective pretrained feature extractor, based on ResNet50, without the need of relying on the complete availability of all the data, with a valuable saving of time and resources.