mlsanityData to 7 Oct 2026

On Privacy and Personalization in Cross-Silo Federated Learning

arXiv 2206.07902 ↗First posted 16 Jun 2022cs.LG cs.CR stat.MLcited by 36

Not on the map. Why?

Abstract (arXiv)

While the application of differential privacy (DP) has been well-studied in cross-device federated learning (FL), there is a lack of work considering DP and its implications for cross-silo FL, a setting characterized by a limited number of clients each containing many data subjects. In cross-silo FL, usual notions of client-level DP are less suitable as real-world privacy regulations typically concern the in-silo data subjects rather than the silos themselves. In this work, we instead consider an alternative notion of silo-specific sample-level DP, where silos set their own privacy targets for their local examples. Under this setting, we reconsider the roles of personalization in federated learning. In particular, we show that mean-regularized multi-task learning (MR-MTL), a simple personalization framework, is a strong baseline for cross-silo FL: under stronger privacy requirements, silos are incentivized to federate more with each other to mitigate DP noise, resulting in consistent improvements relative to standard baseline methods. We provide an empirical study of competing methods as well as a theoretical characterization of MR-MTL for mean estimation, highlighting the interplay between privacy and cross-silo data heterogeneity. Our work serves to establish baselines for private cross-silo FL as well as identify key directions of future work in this area.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 70 of its 128 references that are in the map. First posted on arXiv 16 Jun 2022.

Builds on

70 of 128 references.

References · 70

  1. 2017Attention Is All You Needcited by 59,932
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2022A ConvNet for the 2020scited by 3,545
  7. 2019Advances and Open Problems in Federated Learningcited by 3,021
  8. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  9. 2016Deep Learning with Differential Privacycited by 2,682
  10. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  11. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
  12. 2018Federated Learning with Non-IID Datacited by 1,099
38 more
  1. 2020Adaptive Federated Optimizationcited by 1,036
  2. 2017A Survey on Multi-Task Learningcited by 838
  3. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
  4. 2018LEAF: A Benchmark for Federated Settingscited by 780
  5. 2017Federated Multi-Task Learningcited by 669
  6. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  7. 2017Renyi Differential Privacycited by 643
  8. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  9. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  10. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  11. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  12. 2020Adaptive Personalized Federated Learningcited by 353
  13. 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
  14. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 322
  15. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 321
  16. 2021See through Gradients: Image Batch Recovery via GradInversioncited by 320
  17. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 264
  18. 2019Differentially Private Learning with Adaptive Clippingcited by 218
  19. 2021Personalized Federated Learning using Hypernetworkscited by 210
  20. 2018A Hybrid Approach to Privacy-Preserving Federated Learningcited by 208
  21. 2020Federated Learning of a Mixture of Global and Local Modelscited by 208
  22. 2019R'enyi Differential Privacy of the Sampled Gaussian Mechanismcited by 199
  23. 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
  24. 2018Amplification by Shuffling: From Local to Central Differential Privacy via Anonymitycited by 163
  25. 2019Federated Evaluation of On-device Personalizationcited by 147
  26. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  27. 2020The Discrete Gaussian for Differential Privacycited by 134
  28. 2021The Distributed Discrete Gaussian Mechanism for Federated Learning with Secure Aggregationcited by 127
  29. 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
  30. 2018Distributed Differential Privacy via Shufflingcited by 115
  31. 2021Hyperparameter Tuning with Renyi Differential Privacycited by 104
  32. 2014Deep learning with Elastic Averaging SGDcited by 90
  33. 2018A General Approach to Adding Differential Privacy to Iterative Training Procedurescited by 84
  34. 2019AdaCliP: Adaptive Clipping for Private SGDcited by 77
  35. 2018Private Selection from Private Candidatescited by 75
  36. 2021Learning with User-Level Privacycited by 72
  37. 2022Fishing for User Data in Large-Batch Federated Learning via Gradient Magnificationcited by 72
  38. 2019Differentially Private Model Publishing for Deep Learningcited by 69

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 12; 2024: 9; 2025: 13; 2026: 1.

36 papers.

Sort:

  1. 2023Bayesian Federated Learning: A Surveycited by 22
  2. 2023FedHCA2: Towards Hetero-Client Federated Multi-Task Learningcited by 17
  3. 2023A Survey for Federated Learning Evaluations: Goals and Measurescited by 16
  4. 2023PerAda: Parameter-Efficient Federated Learning Personalization with Generalization Guaranteescited by 15
  5. 2023On Differentially Private Federated Linear Contextual Banditscited by 14
  6. 2022Unraveling the Connections between Privacy and Certified Robustness in Federated Learning Against Poisoning Attackscited by 12
  7. 2025Tackling Feature and Sample Heterogeneity in Decentralized Multi-Task Learning: A Sheaf-Theoretic Approachcited by 10
  8. 2023ULDP-FL: Federated Learning with Across Silo User-Level Differential Privacycited by 9
  9. 2024Free-Rider and Conflict Aware Collaboration Formation for Cross-Silo Federated Learningcited by 9
  10. 2024Private Heterogeneous Federated Learning Without a Trusted Server Revisited: Error-Optimal and Communication-Efficient Algorithms for Convex Lossescited by 8
  11. 2024Personalized Federated Learning with Mixture of Models for Adaptive Prediction and Model Fine-Tuningcited by 8
  12. 2024Cross-silo Federated Learning with Record-level Personalized Differential Privacycited by 7
24 more
  1. 2023Client-Level Differential Privacy via Adaptive Intermediary in Federated Medical Imagingcited by 6
  2. 2024Adaptive Self-Supervised Learning Strategies for Dynamic On-Device LLM Personalizationcited by 5
  3. 2025FedMentor: Domain-Aware Differential Privacy for Heterogeneous Federated LLMs in Mental Healthcited by 5
  4. 2023Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networkscited by 4
  5. 2024Differentially Private Clustered Federated Learningcited by 4
  6. 2025Multi-task Federated Learning with Encoder-Decoder Structure: Enabling Collaborative Learning Across Different Taskscited by 4
  7. 2025Research in Collaborative Learning Does Not Serve Cross-Silo Federated Learning in Practicecited by 3
  8. 2023FedSelect: Customized Selection of Parameters for Fine-Tuning during Personalized Federated Learningcited by 2
  9. 2023Federated Learning is Better with Non-Homomorphic Encryptioncited by 2
  10. 2024Federated Multi-Task Learning on Non-IID Data Silos: An Experimental Studycited by 2
  11. 2025FAST: Federated Active Learning with Foundation Models for Communication-efficient Sampling and Trainingcited by 2
  12. 2025FedMPDD: Communication-Efficient Federated Learning with Privacy Preservation Attributes via Projected Directional Derivativecited by 2
  13. 2023Graph Learning Across Data Siloscited by 1
  14. 2023Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignmentcited by 1
  15. 2024Budgeted Online Model Selection and Fine-Tuning via Federated Learningcited by 1
  16. 2025Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacycited by 1
  17. 2025Enhancing Convergence, Privacy and Fairness for Wireless Personalized Federated Learning: Quantization-Assisted Min-Max Fair Schedulingcited by 1
  18. 2024A Stochastic Optimization Framework for Private and Fair Learning From Decentralized Datacited by 0
  19. 2025Privacy Preserving and Robust Aggregation for Cross-Silo Federated Learning in Non-IID Settingscited by 0
  20. 2025Adaptive Latent-Space Constraints in Personalized Federated Learningcited by 0
  21. 2025Convergence-Privacy-Fairness Trade-Off in Personalized Federated Learningcited by 0
  22. 2025Differentially Private Federated Clustering with Random Rebalancingcited by 0
  23. 2025PubSub-VFL: Towards Efficient Two-Party Split Learning in Heterogeneous Environments via Publisher/Subscriber Architecturecited by 0
  24. 2026MEMOA: Massive Mixtures of Online Agents via Mean-Field Decentralized Nash Equilibriacited by 0