mlsanityData to 7 Oct 2026

MAPL: Model Agnostic Peer-to-peer Learning

arXiv 2403.19792 ↗First posted 28 Mar 2024cs.LG cs.AI cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Effective collaboration among heterogeneous clients in a decentralized setting is a rather unexplored avenue in the literature. To structurally address this, we introduce Model Agnostic Peer-to-peer Learning (coined as MAPL) a novel approach to simultaneously learn heterogeneous personalized models as well as a collaboration graph through peer-to-peer communication among neighboring clients. MAPL is comprised of two main modules: (i) local-level Personalized Model Learning (PML), leveraging a combination of intra- and inter-client contrastive losses; (ii) network-wide decentralized Collaborative Graph Learning (CGL) dynamically refining collaboration weights in a privacy-preserving manner based on local task similarities. Our extensive experimentation demonstrates the efficacy of MAPL and its competitive (or, in most cases, superior) performance compared to its centralized model-agnostic counterparts, without relying on any central server. Our code is available and can be accessed here: https://github.com/SayakMukherjee/MAPL

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 53 of its 72 references that are in the map. First posted on arXiv 28 Mar 2024.

Builds on

53 of 72 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2014Going Deeper with Convolutionscited by 4,231
  8. 2019Advances and Open Problems in Federated Learningcited by 3,021
  9. 2018Federated Optimization in Heterogeneous Networkscited by 2,971
  10. 2020Supervised Contrastive Learningcited by 2,803
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  12. 2021Model-Contrastive Federated Learningcited by 762
38 more
  1. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 678
  2. 2020Ditto: Fair and Robust Federated Learning Through Personalizationcited by 665
  3. 2020Personalized Federated Learning with Moreau Envelopescited by 612
  4. 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
  5. 2019Federated Learning with Personalization Layerscited by 533
  6. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
  7. 2017Can Decentralized Algorithms Outperform Centralized Algorithms? A Case Study for Decentralized Parallel Stochastic Gradient Descentcited by 496
  8. 2021Towards Personalized Federated Learningcited by 462
  9. 2021Data-Free Knowledge Distillation for Heterogeneous Federated Learningcited by 380
  10. 2021FedProto: Federated Prototype Learning across Heterogeneous Clientscited by 349
  11. 2019The Non-IID Data Quagmire of Decentralized Machine Learningcited by 303
  12. 2021Personalized Federated Learning using Hypernetworkscited by 210
  13. 2019A Survey on Distributed Machine Learningcited by 187
  14. 2023Heterogeneous Federated Learning: State-of-the-art and Research Challengescited by 160
  15. 2016How to learn a graph from smooth signalscited by 146
  16. 2021Decentralized Federated Averagingcited by 114
  17. 2022Federated Learning with Partial Model Personalizationcited by 114
  18. 2021Parameterized Knowledge Transfer for Personalized Federated Learningcited by 108
  19. 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
  20. 2021FedProc: Prototypical Contrastive Federated Learning on Non-IID datacited by 80
  21. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  22. 2016Decentralized Collaborative Learning of Personalized Models over Networkscited by 73
  23. 2021Dissecting Supervised Contrastive Learningcited by 64
  24. 2017Collaborative Deep Learning in Fixed Topology Networkscited by 60
  25. 2021Decentralized Federated Learning through Proxy Model Sharingcited by 56
  26. 2023Improving the Model Consistency of Decentralized Federated Learningcited by 45
  27. 2022A Survey on Heterogeneous Federated Learningcited by 42
  28. 2019Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphscited by 34
  29. 2022Beyond spectral gap: The role of the topology in decentralized learningcited by 27
  30. 2022FedClassAvg: Local Representation Learning for Personalized Federated Learning on Heterogeneous Neural Networkscited by 23
  31. 2021Federated Learning with Taskonomy for Non-IID Datacited by 21
  32. 2023Personalized Decentralized Federated Learning with Knowledge Distillationcited by 21
  33. 2021FedH2L: Federated Learning with Model and Statistical Heterogeneitycited by 17
  34. 2022A principled framework for the design and analysis of token algorithmscited by 17
  35. 2020Dual-Free Stochastic Decentralized Optimization with Variance Reductioncited by 12
  36. 2022Does Learning from Decentralized Non-IID Unlabeled Data Benefit from Self Supervision?cited by 12
  37. 2022Completely Heterogeneous Federated Learningcited by 9
  38. 2023pFedSim: Similarity-Aware Model Aggregation Towards Personalized Federated Learningcited by 9

50 most cited of 53 All 53 (JSON)

Built on by

Not cited yet.