mlsanityData to 14 Oct 2026

Privacy and Accuracy Implications of Model Complexity and Integration in Heterogeneous Federated Learning

arXiv 2311.17750 ↗First posted 29 Nov 2023cs.LG cs.AI cs.CRcited by 2

Not on the map. Why?

Abstract (arXiv)

Federated Learning (FL) has been proposed as a privacy-preserving solution for distributed machine learning, particularly in heterogeneous FL settings where clients have varying computational capabilities and thus train models with different complexities compared to the server's model. However, FL is not without vulnerabilities: recent studies have shown that it is susceptible to membership inference attacks (MIA), which can compromise the privacy of client data. In this paper, we examine the intersection of these two aspects, heterogeneous FL and its privacy vulnerabilities, by focusing on the role of client model integration, the process through which the server integrates parameters from clients' smaller models into its larger model. To better understand this process, we first propose a taxonomy that categorizes existing heterogeneous FL methods and enables the design of seven novel heterogeneous FL model integration strategies. Using CIFAR-10, CIFAR-100, and FEMNIST vision datasets, we evaluate the privacy and accuracy trade-offs of these approaches under three types of MIAs. Our findings reveal significant differences in privacy leakage and performance depending on the integration method. Notably, introducing randomness in the model integration process enhances client privacy while maintaining competitive accuracy for both the clients and the server. This work provides quantitative light on the privacy-accuracy implications client model integration in heterogeneous FL settings, paving the way towards more secure and efficient FL systems.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 23 of its 48 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

23 of 48 references.

References · 23

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 864
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  4. 2018LEAF: A Benchmark for Federated Settingscited by 792
  5. 2021Model-Contrastive Federated Learningcited by 766
  6. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 459
  7. 2020Flower: A Friendly Federated Learning Research Frameworkcited by 455
  8. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  9. 2021Membership Inference Attacks on Machine Learning: A Surveycited by 321
  10. 2021FjORD: Fair and Accurate Federated Learning under heterogeneous targets with Ordered Dropoutcited by 192
  11. 2018Expanding the Reach of Federated Learning by Reducing Client Resource Requirementscited by 163
  12. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 161
11 more
  1. 2021Federated Learning for Open Bankingcited by 141
  2. 2020Revisiting Membership Inference Under Realistic Assumptionscited by 112
  3. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 101
  4. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 81
  5. 2021Examining and Combating Spurious Features under Distribution Shiftcited by 65
  6. 2020XOR Mixup: Privacy-Preserving Data Augmentation for One-Shot Federated Learningcited by 58
  7. 2020Differentially Private Secure Multi-Party Computation for Federated Learning in Financial Applicationscited by 58
  8. 2022No One Left Behind: Inclusive Federated Learning over Heterogeneous Devicescited by 37
  9. 2020Heterogeneous Data-Aware Federated Learningcited by 9
  10. 2022A Snapshot of the Frontiers of Client Selection in Federated Learningcited by 9
  11. 2023FedIN: Federated Intermediate Layers Learning for Model Heterogeneitycited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2024Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learningcited by 7
  2. 2025Estimating Model-Level Membership Inference Vulnerability Without Reference Modelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Privacy in Federated Learning
  2. 2024Accuracy-Privacy Trade-off in the Mitigation of Membership Inference Attack in Federated Learning
  3. 2025FedRand: Enhancing Privacy in Federated Learning with Randomized LoRA Subparameter Updates
  4. 2023A Survey of What to Share in Federated Learning: Perspectives on Model Utility, Privacy Leakage, and Communication Efficiency
  5. 2025Federated Learning: A Survey on Privacy-Preserving Collaborative Intelligence
  6. 2022Completely Heterogeneous Federated Learning
  7. 2024Membership Inference Attacks and Defenses in Federated Learning: A Survey
  8. 2023MIA-BAD: An Approach for Enhancing Membership Inference Attack and its Mitigation with Federated Learning