# Incremental Learning for Heterogeneous Structure Segmentation in Brain Tumor MRI

arXiv: https://arxiv.org/abs/2305.19404 · HTML: https://mlsanity.com/paper/2305.19404/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 30 May 2023.

## Builds on (most cited first, 30 of 30)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [Identifying the Best Machine Learning Algorithms for Brain Tumor Segmentation, Progression Assessment, and Overall Survival Prediction in the BRATS Challenge](https://mlsanity.com/paper/1811.02629/) (2018)
- [Dreaming to Distill: Data-free Knowledge Transfer via DeepInversion](https://mlsanity.com/paper/1912.08795/) (2019)
- [Confidence Regularized Self-Training](https://mlsanity.com/paper/1908.09822/) (2019)
- [Embracing Imperfect Datasets: A Review of Deep Learning Solutions for Medical Image Segmentation](https://mlsanity.com/paper/1908.10454/) (2019)
- [Domain-Specific Batch Normalization for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1906.03950/) (2019)
- [PLOP: Learning without Forgetting for Continual Semantic Segmentation](https://mlsanity.com/paper/2011.11390/) (2020)
- [Adaptive Aggregation Networks for Class-Incremental Learning](https://mlsanity.com/paper/2010.05063/) (2020)
- [Batch Renormalization: Towards Reducing Minibatch Dependence in Batch-Normalized Models](https://mlsanity.com/paper/1702.03275/) (2017)
- [Deep Unsupervised Domain Adaptation: A Review of Recent Advances and Perspectives](https://mlsanity.com/paper/2208.07422/) (2022)
- [Representation Compensation Networks for Continual Semantic Segmentation](https://mlsanity.com/paper/2203.05402/) (2022)
- [Reparameterizing Convolutions for Incremental Multi-Task Learning without Task Interference](https://mlsanity.com/paper/2007.12540/) (2020)
- [Adapting Off-the-Shelf Source Segmenter for Target Medical Image Segmentation](https://mlsanity.com/paper/2106.12497/) (2021)
- [Uncertainty-aware Contrastive Distillation for Incremental Semantic Segmentation](https://mlsanity.com/paper/2203.14098/) (2022)
- [Class-Incremental Domain Adaptation](https://mlsanity.com/paper/2008.01389/) (2020)
- [Self-Training for Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2012.03362/) (2020)
- [Unsupervised Domain Adaptation of Black-Box Source Models](https://mlsanity.com/paper/2101.02839/) (2021)
- [Learn the new, keep the old: Extending pretrained models with new anatomy and images](https://mlsanity.com/paper/1806.00265/) (2018)
- [Incremental Learning Meets Transfer Learning: Application to Multi-site Prostate MRI Segmentation](https://mlsanity.com/paper/2206.01369/) (2022)
- [ACT: Semi-supervised Domain-adaptive Medical Image Segmentation with Asymmetric Co-training](https://mlsanity.com/paper/2206.02288/) (2022)
- [Learning Incrementally to Segment Multiple Organs in a CT Image](https://mlsanity.com/paper/2203.02100/) (2022)
- [Domain-incremental Cardiac Image Segmentation with Style-oriented Replay and Domain-sensitive Feature Whitening](https://mlsanity.com/paper/2211.04862/) (2022)
- [Normalization in Training U-Net for 2D Biomedical Semantic Segmentation](https://mlsanity.com/paper/1809.03783/) (2018)
- [Memory Consistent Unsupervised Off-the-Shelf Model Adaptation for Source-Relaxed Medical Image Segmentation](https://mlsanity.com/paper/2209.07910/) (2022)
- [Incremental Learning with Maximum Entropy Regularization: Rethinking Forgetting and Intransigence](https://mlsanity.com/paper/1902.00829/) (2019)
- [Subtype-Aware Dynamic Unsupervised Domain Adaptation](https://mlsanity.com/paper/2208.07754/) (2022)
- [Attentive Continuous Generative Self-training for Unsupervised Domain Adaptive Medical Image Translation](https://mlsanity.com/paper/2305.14589/) (2023)

## Built on by (most cited first, 9 of 9)

- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [Continual Learning in Medical Imaging: A Survey and Practical Analysis](https://mlsanity.com/paper/2405.13482/) (2024)
- [Mixture-of-Shape-Experts (MoSE): End-to-End Shape Dictionary Framework to Prompt SAM for Generalizable Medical Segmentation](https://mlsanity.com/paper/2504.09601/) (2025)
- [Comprehensive Generative Replay for Task-Incremental Segmentation with Concurrent Appearance and Semantic Forgetting](https://mlsanity.com/paper/2406.19796/) (2024)
- [Dynamic Robot-Assisted Surgery with Hierarchical Class-Incremental Semantic Segmentation](https://mlsanity.com/paper/2508.01713/) (2025)
- [Point-supervised Brain Tumor Segmentation with Box-prompted MedSAM](https://mlsanity.com/paper/2408.00706/) (2024)
- [Treatment-wise Glioblastoma Survival Inference with Multi-parametric Preoperative MRI](https://mlsanity.com/paper/2402.06982/) (2024)
- [Privacy-Aware Continual Self-Supervised Learning on Multi-Window Chest Computed Tomography for Domain-Shift Robustness](https://mlsanity.com/paper/2510.27213/) (2025)
- [Multimodal MRI Report Findings Supervised Brain Lesion Segmentation with Substructures](https://mlsanity.com/paper/2602.20994/) (2026)

## Abstract (arXiv)

Deep learning (DL) models for segmenting various anatomical structures have achieved great success via a static DL model that is trained in a single source domain. Yet, the static DL model is likely to perform poorly in a continually evolving environment, requiring appropriate model updates. In an incremental learning setting, we would expect that well-trained static models are updated, following continually evolving target domain data -- e.g., additional lesions or structures of interest -- collected from different sites, without catastrophic forgetting. This, however, poses challenges, due to distribution shifts, additional structures not seen during the initial model training, and the absence of training data in a source domain. To address these challenges, in this work, we seek to progressively evolve an ``off-the-shelf" trained segmentation model to diverse datasets with additional anatomical categories in a unified manner. Specifically, we first propose a divergence-aware dual-flow module with balanced rigidity and plasticity branches to decouple old and new tasks, which is guided by continuous batch renormalization. Then, a complementary pseudo-label training scheme with self-entropy regularized momentum MixUp decay is developed for adaptive network optimization. We evaluated our framework on a brain tumor segmentation task with continually changing target domains -- i.e., new MRI scanners/modalities with incremental structures. Our framework was able to well retain the discriminability of previously learned structures, hence enabling the realistic life-long segmentation model extension along with the widespread accumulation of big medical data.