# Decentralized Collaborative Learning of Personalized Models over Networks

arXiv: https://arxiv.org/abs/1610.05202 · HTML: https://mlsanity.com/paper/1610.05202/

## Key facts

- Cited by 76 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2016.

## Built on by (most cited first, 30 of 76)

- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [Decentralized Federated Learning: Fundamentals, State of the Art, Frameworks, Trends, and Challenges](https://mlsanity.com/paper/2211.08413/) (2022)
- [Federated Evaluation and Tuning for On-Device Personalization: System Design & Applications](https://mlsanity.com/paper/2102.08503/) (2021)
- [From Distributed Machine Learning to Federated Learning: A Survey](https://mlsanity.com/paper/2104.14362/) (2021)
- [Personalized Federated Learning through Local Memorization](https://mlsanity.com/paper/2111.09360/) (2021)
- [Decentralized Federated Learning: Balancing Communication and Computing Costs](https://mlsanity.com/paper/2107.12048/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [Gradient-Leakage Resilient Federated Learning](https://mlsanity.com/paper/2107.01154/) (2021)
- [Personalized Federated Learning under Mixture of Distributions](https://mlsanity.com/paper/2305.01068/) (2023)
- [Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data](https://mlsanity.com/paper/2204.04452/) (2022)
- [A New Look and Convergence Rate of Federated Multi-Task Learning with Laplacian Regularization](https://mlsanity.com/paper/2102.07148/) (2021)
- [FedMoE: Personalized Federated Learning via Heterogeneous Mixture of Experts](https://mlsanity.com/paper/2408.11304/) (2024)
- [Decentralized Local Stochastic Extra-Gradient for Variational Inequalities](https://mlsanity.com/paper/2106.08315/) (2021)
- [Decentralized Federated Learning: A Survey on Security and Privacy](https://mlsanity.com/paper/2401.17319/) (2024)
- [Specialized federated learning using a mixture of experts](https://mlsanity.com/paper/2010.02056/) (2020)
- [Personalization Improves Privacy-Accuracy Tradeoffs in Federated Learning](https://mlsanity.com/paper/2202.05318/) (2022)
- [Preserved central model for faster bidirectional compression in distributed settings](https://mlsanity.com/paper/2102.12528/) (2021)
- [Meta-HAR: Federated Representation Learning for Human Activity Recognition](https://mlsanity.com/paper/2106.00615/) (2021)
- [Machine Learning Practices Outside Big Tech: How Resource Constraints Challenge Responsible Development](https://mlsanity.com/paper/2110.02932/) (2021)
- [Personalized Decentralized Federated Learning with Knowledge Distillation](https://mlsanity.com/paper/2302.12156/) (2023)
- [Self-Aware Personalized Federated Learning](https://mlsanity.com/paper/2204.08069/) (2022)
- [AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devices](https://mlsanity.com/paper/2312.10935/) (2023)
- [D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning](https://mlsanity.com/paper/2104.07365/) (2021)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Opportunistic Federated Learning: An Exploration of Egocentric Collaboration for Pervasive Computing Applications](https://mlsanity.com/paper/2103.13266/) (2021)
- [Backdoor Attacks in Peer-to-Peer Federated Learning](https://mlsanity.com/paper/2301.09732/) (2023)
- [Separation of Powers in Federated Learning](https://mlsanity.com/paper/2105.09400/) (2021)
- [FedVLN: Privacy-preserving Federated Vision-and-Language Navigation](https://mlsanity.com/paper/2203.14936/) (2022)
- [FedDAR: Federated Domain-Aware Representation Learning](https://mlsanity.com/paper/2209.04007/) (2022)
- [Personalized Federated Learning via Heterogeneous Modular Networks](https://mlsanity.com/paper/2210.14830/) (2022)

## Abstract (arXiv)

We consider a set of learning agents in a collaborative peer-to-peer network, where each agent learns a personalized model according to its own learning objective. The question addressed in this paper is: how can agents improve upon their locally trained model by communicating with other agents that have similar objectives? We introduce and analyze two asynchronous gossip algorithms running in a fully decentralized manner. Our first approach, inspired from label propagation, aims to smooth pre-trained local models over the network while accounting for the confidence that each agent has in its initial model. In our second approach, agents jointly learn and propagate their model by making iterative updates based on both their local dataset and the behavior of their neighbors. To optimize this challenging objective, our decentralized algorithm is based on ADMM.