# Addressing catastrophic forgetting for medical domain expansion

arXiv: https://arxiv.org/abs/2103.13511 · HTML: https://mlsanity.com/paper/2103.13511/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 43 references that are in the map.
- First posted on arXiv 24 Mar 2021.

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1802.02611/) (2018)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [EfficientDet: Scalable and Efficient Object Detection](https://mlsanity.com/paper/1911.09070/) (2019)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [Confounding variables can degrade generalization performance of radiological deep learning models](https://mlsanity.com/paper/1807.00431/) (2018)
- [Orthogonal Gradient Descent for Continual Learning](https://mlsanity.com/paper/1910.07104/) (2019)
- [Revisiting Batch Normalization For Practical Domain Adaptation](https://mlsanity.com/paper/1603.04779/) (2016)
- [Continual Learning of Context-dependent Processing in Neural Networks](https://mlsanity.com/paper/1810.01256/) (2018)
- [Automatic diagnosis of the 12-lead ECG using a deep neural network](https://mlsanity.com/paper/1904.01949/) (2019)
- [Towards Robust Evaluations of Continual Learning](https://mlsanity.com/paper/1805.09733/) (2018)
- [Salvaging Federated Learning by Local Adaptation](https://mlsanity.com/paper/2002.04758/) (2020)
- [Federated Learning for Breast Density Classification: A Real-World Implementation](https://mlsanity.com/paper/2009.01871/) (2020)
- [A Lifelong Learning Approach to Brain MR Segmentation Across Scanners and Protocols](https://mlsanity.com/paper/1805.10170/) (2018)
- [The reliability of a deep learning model in clinical out-of-distribution MRI data: a multicohort study](https://mlsanity.com/paper/1911.00515/) (2019)
- [Deep Tone Mapping Operator for High Dynamic Range Images](https://mlsanity.com/paper/1908.04197/) (2019)
- [Dropout as an Implicit Gating Mechanism For Continual Learning](https://mlsanity.com/paper/2004.11545/) (2020)

## Built on by (most cited first, 12 of 12)

- [Rethinking Architecture Design for Tackling Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2106.06047/) (2021)
- [Rethinking Exemplars for Continual Semantic Segmentation in Endoscopy Scenes: Entropy-based Mini-Batch Pseudo-Replay](https://mlsanity.com/paper/2308.14100/) (2023)
- [Continual Hippocampus Segmentation with Transformers](https://mlsanity.com/paper/2204.08043/) (2022)
- [Stabilizing Elastic Weight Consolidation method in practical ML tasks and using weight importances for neural network pruning](https://mlsanity.com/paper/2109.10021/) (2021)
- [Three Applications of Conformal Prediction for Rating Breast Density in Mammography](https://mlsanity.com/paper/2206.12008/) (2022)
- [MoCo-Transfer: Investigating out-of-distribution contrastive learning for limited-data domains](https://mlsanity.com/paper/2311.09401/) (2023)
- [Deploying clinical machine learning? Consider the following...](https://mlsanity.com/paper/2109.06919/) (2021)
- [Heterogeneous Federated Learning with Splited Language Model](https://mlsanity.com/paper/2403.16050/) (2024)
- [CSTRL: Context-Driven Sequential Transfer Learning for Abstractive Radiology Report Summarization](https://mlsanity.com/paper/2503.05750/) (2025)
- [Empirical investigations on WVA structural issues](https://mlsanity.com/paper/2208.05791/) (2022)
- [Correlation of the importances of neural network weights calculated by modern methods of overcoming catastrophic forgetting](https://mlsanity.com/paper/2211.17012/) (2022)
- [Fair Evaluation of Federated Learning Algorithms for Automated Breast Density Classification: The Results of the 2022 ACR-NCI-NVIDIA Federated Learning Challenge](https://mlsanity.com/paper/2405.14900/) (2024)

## Abstract (arXiv)

Model brittleness is a key concern when deploying deep learning models in real-world medical settings. A model that has high performance at one institution may suffer a significant decline in performance when tested at other institutions. While pooling datasets from multiple institutions and retraining may provide a straightforward solution, it is often infeasible and may compromise patient privacy. An alternative approach is to fine-tune the model on subsequent institutions after training on the original institution. Notably, this approach degrades model performance at the original institution, a phenomenon known as catastrophic forgetting. In this paper, we develop an approach to address catastrophic forget-ting based on elastic weight consolidation combined with modulation of batch normalization statistics under two scenarios: first, for expanding the domain from one imaging system's data to another imaging system's, and second, for expanding the domain from a large multi-institutional dataset to another single institution dataset. We show that our approach outperforms several other state-of-the-art approaches and provide theoretical justification for the efficacy of batch normalization modulation. The results of this study are generally applicable to the deployment of any clinical deep learning model which requires domain expansion.