# Differentially private federated deep learning for multi-site medical image segmentation

arXiv: https://arxiv.org/abs/2107.02586 · HTML: https://mlsanity.com/paper/2107.02586/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 6 Jul 2021.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Membership Inference Attacks against Machine Learning Models](https://mlsanity.com/paper/1610.05820/) (2016)
- [Federated Learning: Strategies for Improving Communication Efficiency](https://mlsanity.com/paper/1610.05492/) (2016)
- [Deep Leakage from Gradients](https://mlsanity.com/paper/1906.08935/) (2019)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [iDLG: Improved Deep Leakage from Gradients](https://mlsanity.com/paper/2001.02610/) (2020)
- [A large annotated medical image dataset for the development and evaluation of segmentation algorithms](https://mlsanity.com/paper/1902.09063/) (2019)
- [Differential Privacy Has Disparate Impact on Model Accuracy](https://mlsanity.com/paper/1905.12101/) (2019)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Privacy-preserving Federated Brain Tumour Segmentation](https://mlsanity.com/paper/1910.00962/) (2019)
- [Multi-Institutional Deep Learning Modeling Without Sharing Patient Data: A Feasibility Study on Brain Tumor Segmentation](https://mlsanity.com/paper/1810.04304/) (2018)
- [In Defense of Pre-trained ImageNet Architectures for Real-time Semantic Segmentation of Road-driving Images](https://mlsanity.com/paper/1903.08469/) (2019)
- [Channel Normalization in Convolutional Neural Network avoids Vanishing Gradients](https://mlsanity.com/paper/1907.09539/) (2019)
- [Decentralized Differentially Private Segmentation with PATE](https://mlsanity.com/paper/2004.06567/) (2020)

## Built on by (most cited first, 12 of 12)

- [Differentially Private Diffusion Models](https://mlsanity.com/paper/2210.09929/) (2022)
- [Federated Learning for Computer Vision](https://mlsanity.com/paper/2308.13558/) (2023)
- [From Challenges and Pitfalls to Recommendations and Opportunities: Implementing Federated Learning in Healthcare](https://mlsanity.com/paper/2409.09727/) (2024)
- [Beyond Gradients: Exploiting Adversarial Priors in Model Inversion Attacks](https://mlsanity.com/paper/2203.00481/) (2022)
- [Client-Level Differential Privacy via Adaptive Intermediary in Federated Medical Imaging](https://mlsanity.com/paper/2307.12542/) (2023)
- [Efficient Hyperparameter Optimization for Differentially Private Deep Learning](https://mlsanity.com/paper/2108.03888/) (2021)
- [Analysing the Influence of Attack Configurations on the Reconstruction of Medical Images in Federated Learning](https://mlsanity.com/paper/2204.13808/) (2022)
- [SmoothNets: Optimizing CNN architecture design for differentially private deep learning](https://mlsanity.com/paper/2205.04095/) (2022)
- [Federated nnU-Net for Privacy-Preserving Medical Image Segmentation](https://mlsanity.com/paper/2503.02549/) (2025)
- [Distributed Machine Learning and the Semblance of Trust](https://mlsanity.com/paper/2112.11040/) (2021)
- [A Novel Neural Network-Based Federated Learning System for Imbalanced and Non-IID Data](https://mlsanity.com/paper/2311.10025/) (2023)
- [Federated Continual Learning for Privacy-Preserving Hospital Imaging Classification](https://mlsanity.com/paper/2601.06742/) (2026)

## Abstract (arXiv)

Collaborative machine learning techniques such as federated learning (FL) enable the training of models on effectively larger datasets without data transfer. Recent initiatives have demonstrated that segmentation models trained with FL can achieve performance similar to locally trained models. However, FL is not a fully privacy-preserving technique and privacy-centred attacks can disclose confidential patient data. Thus, supplementing FL with privacy-enhancing technologies (PTs) such as differential privacy (DP) is a requirement for clinical applications in a multi-institutional setting. The application of PTs to FL in medical imaging and the trade-offs between privacy guarantees and model utility, the ramifications on training performance and the susceptibility of the final models to attacks have not yet been conclusively investigated. Here we demonstrate the first application of differentially private gradient descent-based FL on the task of semantic segmentation in computed tomography. We find that high segmentation performance is possible under strong privacy guarantees with an acceptable training time penalty. We furthermore demonstrate the first successful gradient-based model inversion attack on a semantic segmentation model and show that the application of DP prevents it from divulging sensitive image features.