mlsanityData to 7 Oct 2026

P4: Towards private, personalized, and Peer-to-Peer learning

arXiv 2405.17697 ↗First posted 27 May 2024cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Personalized learning is a proposed approach to address the problem of data heterogeneity in collaborative machine learning. In a decentralized setting, the two main challenges of personalization are client clustering and data privacy. In this paper, we address these challenges by developing P4 (Personalized Private Peer-to-Peer) a method that ensures that each client receives a personalized model while maintaining differential privacy guarantee of each client's local dataset during and after the training. Our approach includes the design of a lightweight algorithm to identify similar clients and group them in a private, peer-to-peer (P2P) manner. Once grouped, we develop differentially-private knowledge distillation for clients to co-train with minimal impact on accuracy. We evaluate our proposed method on three benchmark datasets (FEMNIST or Federated EMNIST, CIFAR-10 and CIFAR-100) and two different neural network architectures (Linear and CNN-based networks) across a range of privacy parameters. The results demonstrate the potential of P4, as it outperforms the state-of-the-art of differential private P2P by up to 40 percent in terms of accuracy. We also show the practicality of P4 by implementing it on resource constrained devices, and validating that it has minimal overhead, e.g., about 7 seconds to run collaborative training between two clients.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 30 of its 58 references that are in the map. First posted on arXiv 27 May 2024.

Builds on

30 of 58 references.

References · 30

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,404
  3. 2020Inverting Gradients -- How easy is it to break privacy in federated learning?cited by 707
  4. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  5. 2017Renyi Differential Privacycited by 643
  6. 2017Deep Mutual Learningcited by 621
  7. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  8. 2020An Efficient Framework for Clustered Federated Learningcited by 505
  9. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  10. 2017Differentially Private Federated Learning: A Client Level Perspectivecited by 451
  11. 2019Differential Privacy Has Disparate Impact on Model Accuracycited by 316
  12. 2020Differentially Private Learning Needs Better Features (or Much More Data)cited by 233
18 more
  1. 2020Federated learning with hierarchical clustering of local updates to improve training on non-IID datacited by 222
  2. 2020Personalized Federated Learning with First Order Model Optimizationcited by 201
  3. 2021Evaluating Gradient Inversion Attacks and Defenses in Federated Learningcited by 196
  4. 2021Differentially Private Federated Learning on Heterogeneous Datacited by 74
  5. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  6. 2019On-Device Machine Learning: An Algorithms and Learning Theory Perspectivecited by 63
  7. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  8. 2017Personalized and Private Peer-to-Peer Machine Learningcited by 54
  9. 2013Deep Scattering Spectrumcited by 52
  10. 2014Deep Roto-Translation Scattering for Object Classificationcited by 49
  11. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  12. 2023FedML-HE: An Efficient Homomorphic-Encryption-Based Privacy-Preserving Federated Learning Systemcited by 38
  13. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 34
  14. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
  15. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  16. 2019Asynchronous Federated Learning with Differential Privacy for Edge Intelligencecited by 16
  17. 2023Towards More Suitable Personalization in Federated Learning via Decentralized Partial Model Trainingcited by 8
  18. 2023Generalizing Differentially Private Decentralized Deep Learning with Multi-Agent Consensuscited by 1

Built on by

Not cited yet.