# Decentralized Federated Learning through Proxy Model Sharing

arXiv: https://arxiv.org/abs/2111.11343 · HTML: https://mlsanity.com/paper/2111.11343/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 79 references that are in the map.
- First posted on arXiv 22 Nov 2021.

## Builds on (most cited first, 30 of 43)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Deep Learning with Differential Privacy](https://mlsanity.com/paper/1607.00133/) (2016)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Group Normalization](https://mlsanity.com/paper/1803.08494/) (2018)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Renyi Differential Privacy](https://mlsanity.com/paper/1702.07476/) (2017)
- [Deep Mutual Learning](https://mlsanity.com/paper/1706.00384/) (2017)
- [Exploiting Unintended Feature Leakage in Collaborative Learning](https://mlsanity.com/paper/1805.04049/) (2018)
- [Bayesian Nonparametric Federated Learning of Neural Networks](https://mlsanity.com/paper/1905.12022/) (2019)
- [Analyzing Federated Learning through an Adversarial Lens](https://mlsanity.com/paper/1811.12470/) (2018)
- [Semi-supervised Knowledge Transfer for Deep Learning from Private Training Data](https://mlsanity.com/paper/1610.05755/) (2016)
- [Personalized Cross-Silo Federated Learning on Non-IID Data](https://mlsanity.com/paper/2007.03797/) (2020)
- [Distributed learning of deep neural network over multiple agents](https://mlsanity.com/paper/1810.06060/) (2018)
- [Auditing Differentially Private Machine Learning: How Private is Private SGD?](https://mlsanity.com/paper/2006.07709/) (2020)
- [R'enyi Differential Privacy of the Sampled Gaussian Mechanism](https://mlsanity.com/paper/1908.10530/) (2019)
- [Adversary Instantiation: Lower Bounds for Differentially Private Machine Learning](https://mlsanity.com/paper/2101.04535/) (2021)
- [Privacy-preserving Federated Brain Tumour Segmentation](https://mlsanity.com/paper/1910.00962/) (2019)
- [Stochastic Gradient Push for Distributed Deep Learning](https://mlsanity.com/paper/1811.10792/) (2018)
- [Protection Against Reconstruction and Its Applications in Private Federated Learning](https://mlsanity.com/paper/1812.00984/) (2018)
- [Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization](https://mlsanity.com/paper/1709.08765/) (2017)
- [Multi-site fMRI Analysis Using Privacy-preserving Federated Learning and Domain Adaptation: ABIDE Results](https://mlsanity.com/paper/2001.05647/) (2020)

## Built on by (most cited first, 30 of 57)

- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022)
- [Deep Model Fusion: A Survey](https://mlsanity.com/paper/2309.15698/) (2023)
- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [Federated Learning Enables Big Data for Rare Cancer Boundary Detection](https://mlsanity.com/paper/2204.10836/) (2022)
- [Byzantine-Robust Decentralized Federated Learning](https://mlsanity.com/paper/2406.10416/) (2024)
- [Disparate Impact in Differential Privacy from Gradient Misalignment](https://mlsanity.com/paper/2206.07737/) (2022)
- [A Survey on Decentralized Federated Learning](https://mlsanity.com/paper/2308.04604/) (2023)
- [Federated Model Heterogeneous Matryoshka Representation Learning](https://mlsanity.com/paper/2406.00488/) (2024)
- [FedMM: Federated Multi-Modal Learning with Modality Heterogeneity in Computational Pathology](https://mlsanity.com/paper/2402.15858/) (2024)
- [Self-supervised Representation Learning From Random Data Projectors](https://mlsanity.com/paper/2310.07756/) (2023)
- [Threats and Defenses in Federated Learning Life Cycle: A Comprehensive Survey and Challenges](https://mlsanity.com/paper/2407.06754/) (2024)
- [Advances in Robust Federated Learning: A Survey with Heterogeneity Considerations](https://mlsanity.com/paper/2405.09839/) (2024)
- [Find Your Friends: Personalized Federated Learning with the Right Collaborators](https://mlsanity.com/paper/2210.06597/) (2022)
- [Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation](https://mlsanity.com/paper/2406.12815/) (2024)
- [Tackling Data Heterogeneity in Federated Learning via Loss Decomposition](https://mlsanity.com/paper/2408.12300/) (2024)
- [Personalized and privacy-preserving federated heterogeneous medical image analysis with PPPML-HMI](https://mlsanity.com/paper/2302.11571/) (2023)
- [Stochastic Unrolled Federated Learning](https://mlsanity.com/paper/2305.15371/) (2023)
- [A survey on secure decentralized optimization and learning](https://mlsanity.com/paper/2408.08628/) (2024)
- [Toward Malicious Clients Detection in Federated Learning](https://mlsanity.com/paper/2505.09110/) (2025)
- [Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives](https://mlsanity.com/paper/2307.02140/) (2023)
- [FedSODA: Federated Cross-assessment and Dynamic Aggregation for Histopathology Segmentation](https://mlsanity.com/paper/2312.12824/) (2023)
- [Uncertainty-Based Extensible Codebook for Discrete Federated Learning in Heterogeneous Data Silos](https://mlsanity.com/paper/2402.18888/) (2024)
- [Review of Mathematical Optimization in Federated Learning](https://mlsanity.com/paper/2412.01630/) (2024)
- [Sparse Decentralized Federated Learning](https://mlsanity.com/paper/2308.16671/) (2023)
- [FedEmb: A Vertical and Hybrid Federated Learning Algorithm using Network And Feature Embedding Aggregation](https://mlsanity.com/paper/2312.00102/) (2023)
- [Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2402.12876/) (2024)
- [Practical Insights into Knowledge Distillation for Pre-Trained Models](https://mlsanity.com/paper/2402.14922/) (2024)
- [Cyclical Weight Consolidation: Towards Solving Catastrophic Forgetting in Serial Federated Learning](https://mlsanity.com/paper/2405.10647/) (2024)
- [Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Review](https://mlsanity.com/paper/2409.16340/) (2024)

## Abstract (arXiv)

Institutions in highly regulated domains such as finance and healthcare often have restrictive rules around data sharing. Federated learning is a distributed learning framework that enables multi-institutional collaborations on decentralized data with improved protection for each collaborator's data privacy. In this paper, we propose a communication-efficient scheme for decentralized federated learning called ProxyFL, or proxy-based federated learning. Each participant in ProxyFL maintains two models, a private model, and a publicly shared proxy model designed to protect the participant's privacy. Proxy models allow efficient information exchange among participants without the need of a centralized server. The proposed method eliminates a significant limitation of canonical federated learning by allowing model heterogeneity; each participant can have a private model with any architecture. Furthermore, our protocol for communication by proxy leads to stronger privacy guarantees using differential privacy analysis. Experiments on popular image datasets, and a cancer diagnostic problem using high-quality gigapixel histology whole slide images, show that ProxyFL can outperform existing alternatives with much less communication overhead and stronger privacy.