# Distributed Weight Consolidation: A Brain Segmentation Case Study

arXiv: https://arxiv.org/abs/1805.10863 · HTML: https://mlsanity.com/paper/1805.10863/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2018.

## Built on by (most cited first, 9 of 9)

- [Subspace Regularizers for Few-Shot Class Incremental Learning](https://mlsanity.com/paper/2110.07059/) (2021)
- [Continual Learning in Medical Image Analysis: A Comprehensive Review of Recent Advancements and Future Prospects](https://mlsanity.com/paper/2312.17004/) (2023)
- [Continual Hippocampus Segmentation with Transformers](https://mlsanity.com/paper/2204.08043/) (2022)
- [Adversary Aware Continual Learning](https://mlsanity.com/paper/2304.14483/) (2023)
- [Federated Bayesian Deep Learning: The Application of Statistical Aggregation Methods to Bayesian Models](https://mlsanity.com/paper/2403.15263/) (2024)
- [Targeted Gradient Descent: A Novel Method for Convolutional Neural Networks Fine-tuning and Online-learning](https://mlsanity.com/paper/2109.14729/) (2021)
- [CILIATE: Towards Fairer Class-based Incremental Learning by Dataset and Training Refinement](https://mlsanity.com/paper/2304.04222/) (2023)
- [VI-PANN: Harnessing Transfer Learning and Uncertainty-Aware Variational Inference for Improved Generalization in Audio Pattern Recognition](https://mlsanity.com/paper/2401.05531/) (2024)
- [From Lab to Pocket: A Novel Continual Learning-based Mobile Application for Screening COVID-19](https://mlsanity.com/paper/2410.12589/) (2024)

## Abstract (arXiv)

Collecting the large datasets needed to train deep neural networks can be very difficult, particularly for the many applications for which sharing and pooling data is complicated by practical, ethical, or legal concerns. However, it may be the case that derivative datasets or predictive models developed within individual sites can be shared and combined with fewer restrictions. Training on distributed data and combining the resulting networks is often viewed as continual learning, but these methods require networks to be trained sequentially. In this paper, we introduce distributed weight consolidation (DWC), a continual learning method to consolidate the weights of separate neural networks, each trained on an independent dataset. We evaluated DWC with a brain segmentation case study, where we consolidated dilated convolutional neural networks trained on independent structural magnetic resonance imaging (sMRI) datasets from different sites. We found that DWC led to increased performance on test sets from the different sites, while maintaining generalization performance for a very large and completely independent multi-site dataset, compared to an ensemble baseline.