# Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening

arXiv: https://arxiv.org/abs/2211.04862 · HTML: https://mlsanity.com/paper/2211.04862/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 41 references that are in the map.
- First posted on arXiv 9 Nov 2022.

## Builds on (most cited first, 30 of 33)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [FedBN: Federated Learning on Non-IID Features via Local Batch Normalization](https://mlsanity.com/paper/2102.07623/) (2021)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net](https://mlsanity.com/paper/1807.09441/) (2018)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Gradient Projection Memory for Continual Learning](https://mlsanity.com/paper/2103.09762/) (2021)
- [Modeling the Background for Incremental Learning in Semantic Segmentation](https://mlsanity.com/paper/2002.00718/) (2020)
- [Federated Continual Learning with Weighted Inter-client Transfer](https://mlsanity.com/paper/2003.03196/) (2020)
- [Deep learning for cardiac image segmentation: A review](https://mlsanity.com/paper/1911.03723/) (2019)
- [Scalable and Order-robust Continual Learning with Additive Parameter Decomposition](https://mlsanity.com/paper/1902.09432/) (2019)
- [MS-Net: Multi-Site Network for Improving Prostate Segmentation with Heterogeneous MRI Data](https://mlsanity.com/paper/2002.03366/) (2020)
- [Evaluation of Algorithms for Multi-Modality Whole Heart Segmentation: An Open-Access Grand Challenge](https://mlsanity.com/paper/1902.07880/) (2019)
- [Model Complexity of Deep Learning: A Survey](https://mlsanity.com/paper/2103.05127/) (2021)
- [A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols](https://mlsanity.com/paper/1805.10170/) (2018)
- [Image-to-Image Translation via Group-wise Deep Whitening-and-Coloring Transformation](https://mlsanity.com/paper/1812.09912/) (2018)
- [Cardiac Segmentation on Late Gadolinium Enhancement MRI: A Benchmark Study from Multi-Sequence Cardiac MR Segmentation Challenge](https://mlsanity.com/paper/2006.12434/) (2020)
- [What is Wrong with Continual Learning in Medical Image Segmentation?](https://mlsanity.com/paper/2010.11008/) (2020)
- [Learn the new, keep the old: Extending pretrained models with new anatomy and images](https://mlsanity.com/paper/1806.00265/) (2018)
- [Towards continual learning in medical imaging](https://mlsanity.com/paper/1811.02496/) (2018)
- [Importance Driven Continual Learning for Segmentation Across Domains](https://mlsanity.com/paper/2005.00079/) (2020)
- [Dynamic memory to alleviate catastrophic forgetting in continuous learning settings](https://mlsanity.com/paper/2007.02639/) (2020)
- [Incremental Learning for Multi-organ Segmentation with Partially Labeled Datasets](https://mlsanity.com/paper/2103.04526/) (2021)
- [Continual Active Learning for Efficient Adaptation of Machine Learning Models to Changing Image Acquisition](https://mlsanity.com/paper/2106.03351/) (2021)

## Built on by (most cited first, 16 of 16)

- [Federated Domain Generalization: A Survey](https://mlsanity.com/paper/2306.01334/) (2023)
- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [A Survey on Continual Semantic Segmentation: Theory, Challenge, Method and Application](https://mlsanity.com/paper/2310.14277/) (2023)
- [Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI](https://mlsanity.com/paper/2305.19404/) (2023)
- [Continual Learning in Medical Imaging: A Survey and Practical Analysis](https://mlsanity.com/paper/2405.13482/) (2024)
- [Generative appearance replay for continual unsupervised domain adaptation](https://mlsanity.com/paper/2301.01211/) (2023)
- [Multi-View Class Incremental Learning](https://mlsanity.com/paper/2306.09675/) (2023)
- [A Survey of Incremental Transfer Learning: Combining Peer-to-Peer Federated Learning and Domain Incremental Learning for Multicenter Collaboration](https://mlsanity.com/paper/2309.17192/) (2023)
- [Memory-Efficient Prompt Tuning for Incremental Histopathology Classification](https://mlsanity.com/paper/2401.11674/) (2024)
- [Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting](https://mlsanity.com/paper/2406.19796/) (2024)
- [Towards Synchronous Memorizability and Generalizability with Site-Modulated Diffusion Replay for Cross-Site Continual Segmentation](https://mlsanity.com/paper/2406.18037/) (2024)
- [Unlocking Positive Transfer in Incrementally Learning Surgical Instruments: A Self-reflection Hierarchical Prompt Framework](https://mlsanity.com/paper/2604.02877/) (2026)
- [Dual-Alignment Knowledge Retention for Continual Medical Image Segmentation](https://mlsanity.com/paper/2507.03638/) (2025)
- [Investigating ECG Diagnosis with Ambiguous Labels using Partial Label Learning](https://mlsanity.com/paper/2512.11095/) (2025)
- [C^2GR: Coupled Comprehensive Generative Replay for a Continually Learnable Universal Segmentation Model](https://mlsanity.com/paper/2606.23473/) (2026)
- [Learning When to Recur: Token-Adaptive Recursion for Imbalanced Ophthalmic Domain Incremental Learning](https://mlsanity.com/paper/2609.32785/) (2026)

## Abstract (arXiv)

Contemporary methods have shown promising results on cardiac image segmentation, but merely in static learning, i.e., optimizing the network once for all, ignoring potential needs for model updating. In real-world scenarios, new data continues to be gathered from multiple institutions over time and new demands keep growing to pursue more satisfying performance. The desired model should incrementally learn from each incoming dataset and progressively update with improved functionality as time goes by. As the datasets sequentially delivered from multiple sites are normally heterogenous with domain discrepancy, each updated model should not catastrophically forget previously learned domains while well generalizing to currently arrived domains or even unseen domains. In medical scenarios, this is particularly challenging as accessing or storing past data is commonly not allowed due to data privacy. To this end, we propose a novel domain-incremental learning framework to recover past domain inputs first and then regularly replay them during model optimization. Particularly, we first present a style-oriented replay module to enable structure-realistic and memory-efficient reproduction of past data, and then incorporate the replayed past data to jointly optimize the model with current data to alleviate catastrophic forgetting. During optimization, we additionally perform domain-sensitive feature whitening to suppress model's dependency on features that are sensitive to domain changes (e.g., domain-distinctive style features) to assist domain-invariant feature exploration and gradually improve the generalization performance of the network. We have extensively evaluated our approach with the M&Ms Dataset in single-domain and compound-domain incremental learning settings with improved performance over other comparison approaches.