# Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learning

arXiv: https://arxiv.org/abs/2311.17750 · HTML: https://mlsanity.com/paper/2311.17750/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 48 references that are in the map.
- First posted on arXiv 29 Nov 2023.

## Builds on (most cited first, 23 of 23)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks](https://mlsanity.com/paper/1802.08232/) (2018)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Flower: A Friendly Federated Learning Research Framework](https://mlsanity.com/paper/2007.14390/) (2020)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Membership Inference Attacks on Machine Learning: A Survey](https://mlsanity.com/paper/2103.07853/) (2021)
- [FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropout](https://mlsanity.com/paper/2102.13451/) (2021)
- [Expanding the Reach of Federated Learning by Reducing Client Resource Requirements](https://mlsanity.com/paper/1812.07210/) (2018)
- [Heterogeneous Federated Learning: State-of-the-art and Research Challenges](https://mlsanity.com/paper/2307.10616/) (2023)
- [Federated Learning for Open Banking](https://mlsanity.com/paper/2108.10749/) (2021)
- [Revisiting Membership Inference Under Realistic Assumptions](https://mlsanity.com/paper/2005.10881/) (2020)
- [Membership Inference Attacks by Exploiting Loss Trajectory](https://mlsanity.com/paper/2208.14933/) (2022)
- [FedProc: Prototypical Contrastive Federated Learning on Non-IID data](https://mlsanity.com/paper/2109.12273/) (2021)
- [Examining and Combating Spurious Features under Distribution Shift](https://mlsanity.com/paper/2106.07171/) (2021)
- [XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learning](https://mlsanity.com/paper/2006.05148/) (2020)
- [Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applications](https://mlsanity.com/paper/2010.05867/) (2020)
- [No One Left Behind: Inclusive Federated Learning over Heterogeneous Devices](https://mlsanity.com/paper/2202.08036/) (2022)
- [Heterogeneous Data-Aware Federated Learning](https://mlsanity.com/paper/2011.06393/) (2020)
- [A Snapshot of the Frontiers of Client Selection in Federated Learning](https://mlsanity.com/paper/2210.04607/) (2022)
- [FedIN: Federated Intermediate Layers Learning for Model Heterogeneity](https://mlsanity.com/paper/2304.00759/) (2023)

## Built on by (most cited first, 2 of 2)

- [Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning](https://mlsanity.com/paper/2402.06674/) (2024)
- [Estimating Model-Level Membership Inference Vulnerability Without Reference Models](https://mlsanity.com/paper/2510.19773/) (2025)

## Abstract (arXiv)

Federated Learning (FL) has been proposed as a privacy-preserving solution for distributed machine learning, particularly in heterogeneous FL settings where clients have varying computational capabilities and thus train models with different complexities compared to the server's model. However, FL is not without vulnerabilities: recent studies have shown that it is susceptible to membership inference attacks (MIA), which can compromise the privacy of client data. In this paper, we examine the intersection of these two aspects, heterogeneous FL and its privacy vulnerabilities, by focusing on the role of client model integration, the process through which the server integrates parameters from clients' smaller models into its larger model. To better understand this process, we first propose a taxonomy that categorizes existing heterogeneous FL methods and enables the design of seven novel heterogeneous FL model integration strategies. Using CIFAR-10, CIFAR-100, and FEMNIST vision datasets, we evaluate the privacy and accuracy trade-offs of these approaches under three types of MIAs. Our findings reveal significant differences in privacy leakage and performance depending on the integration method. Notably, introducing randomness in the model integration process enhances client privacy while maintaining competitive accuracy for both the clients and the server. This work provides quantitative light on the privacy-accuracy implications client model integration in heterogeneous FL settings, paving the way towards more secure and efficient FL systems.