# Dual-Alignment Knowledge Retention for Continual Medical Image Segmentation

arXiv: https://arxiv.org/abs/2507.03638 · HTML: https://mlsanity.com/paper/2507.03638/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 50 references that are in the map.
- First posted on arXiv 4 Jul 2025.

## Builds on (most cited first, 29 of 29)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Memory Aware Synapses: Learning what (not) to forget](https://mlsanity.com/paper/1711.09601/) (2017)
- [Dark Experience for General Continual Learning: a Strong, Simple Baseline](https://mlsanity.com/paper/2004.07211/) (2020)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [DER: Dynamically Expandable Representation for Class Incremental Learning](https://mlsanity.com/paper/2103.16788/) (2021)
- [FedDG: Federated Domain Generalization on Medical Image Segmentation via Episodic Learning in Continuous Frequency Space](https://mlsanity.com/paper/2103.06030/) (2021)
- [Selective Experience Replay for Lifelong Learning](https://mlsanity.com/paper/1802.10269/) (2018)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Shape-aware Meta-learning for Generalizing Prostate MRI Segmentation to Unseen Domains](https://mlsanity.com/paper/2007.02035/) (2020)
- [Continual Deep Learning by Functional Regularisation of Memorable Past](https://mlsanity.com/paper/2004.14070/) (2020)
- [Continual Learning for Abdominal Multi-Organ and Tumor Segmentation](https://mlsanity.com/paper/2306.00988/) (2023)
- [Meta-attention for ViT-backed Continual Learning](https://mlsanity.com/paper/2203.11684/) (2022)
- [LifeLonger: A Benchmark for Continual Disease Classification](https://mlsanity.com/paper/2204.05737/) (2022)
- [On Generalizing Beyond Domains in Cross-Domain Continual Learning](https://mlsanity.com/paper/2203.03970/) (2022)
- [Towards continual learning in medical imaging](https://mlsanity.com/paper/1811.02496/) (2018)
- [Importance Driven Continual Learning for Segmentation Across Domains](https://mlsanity.com/paper/2005.00079/) (2020)
- [Continual Segment: Towards a Single, Unified and Accessible Continual Segmentation Model of 143 Whole-body Organs in CT Scans](https://mlsanity.com/paper/2302.00162/) (2023)
- [Contrastive Semi-supervised Learning for Domain Adaptive Segmentation Across Similar Anatomical Structures](https://mlsanity.com/paper/2208.08605/) (2022)
- [Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening](https://mlsanity.com/paper/2211.04862/) (2022)
- [DualHSIC: HSIC-Bottleneck and Alignment for Continual Learning](https://mlsanity.com/paper/2305.00380/) (2023)
- [Learning with Style: Continual Semantic Segmentation Across Tasks and Domains](https://mlsanity.com/paper/2210.07016/) (2022)
- [Leveraging Old Knowledge to Continually Learn New Classes in Medical Images](https://mlsanity.com/paper/2303.13752/) (2023)
- [Learning to Predict Gradients for Semi-Supervised Continual Learning](https://mlsanity.com/paper/2201.09196/) (2022)
- [Learning to Learn for Few-shot Continual Active Learning](https://mlsanity.com/paper/2311.03732/) (2023)
- [Class Impression for Data-free Incremental Learning](https://mlsanity.com/paper/2207.00005/) (2022)

## Abstract (arXiv)

Continual learning in medical image segmentation involves sequential data acquisition across diverse domains (e.g., clinical sites), where task interference between past and current domains often leads to catastrophic forgetting. Existing continual learning methods fail to capture the complex dependencies between tasks. We introduce a novel framework that mitigates forgetting by establishing and enhancing complex dependencies between historical data and the network in the present task. Our framework features a dual-alignment strategy, the cross-network alignment (CNA) module aligns the features extracted from the bottleneck layers of the current and previous networks, respectively, while the cross-representation alignment (CRA) module aligns the features learned by the current network from historical buffered data and current input data, respectively. Implementing both types of alignment is a non-trivial task. To address this, we further analyze the linear and nonlinear forms of the well-established Hilbert-Schmidt Independence Criterion (HSIC) and deliberately design feature mapping and feature pairing blocks within the CRA module. Experiments on medical image segmentation task demonstrate our framework's effectiveness in mitigating catastrophic forgetting under domain shifts.