# Fully Decentralized Joint Learning of Personalized Models and Collaboration Graphs

arXiv: https://arxiv.org/abs/1901.08460 · HTML: https://mlsanity.com/paper/1901.08460/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2019.

## Built on by (most cited first, 30 of 36)

- [Federated Multi-Task Learning under a Mixture of Distributions](https://mlsanity.com/paper/2108.10252/) (2021)
- [Personalized Federated Learning through Local Memorization](https://mlsanity.com/paper/2111.09360/) (2021)
- [Recent Advances on Federated Learning: A Systematic Survey](https://mlsanity.com/paper/2301.01299/) (2023)
- [Refined Convergence and Topology Learning for Decentralized SGD with Heterogeneous Data](https://mlsanity.com/paper/2204.04452/) (2022)
- [Personalized Decentralized Federated Learning with Knowledge Distillation](https://mlsanity.com/paper/2302.12156/) (2023)
- [FedBone: Towards Large-Scale Federated Multi-Task Learning](https://mlsanity.com/paper/2306.17465/) (2023)
- [Multi-level Personalized Federated Learning on Heterogeneous and Long-Tailed Data](https://mlsanity.com/paper/2405.06413/) (2024)
- [D-Cliques: Compensating for Data Heterogeneity with Topology in Decentralized Federated Learning](https://mlsanity.com/paper/2104.07365/) (2021)
- [Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data](https://mlsanity.com/paper/2203.01752/) (2022)
- [FedDAR: Federated Domain-Aware Representation Learning](https://mlsanity.com/paper/2209.04007/) (2022)
- [Beyond the Federation: Topology-aware Federated Learning for Generalization to Unseen Clients](https://mlsanity.com/paper/2407.04949/) (2024)
- [Distributed Learning over Networks with Graph-Attention-Based Personalization](https://mlsanity.com/paper/2305.13041/) (2023)
- [Worldwide Federated Training of Language Models](https://mlsanity.com/paper/2405.14446/) (2024)
- [CoBo: Collaborative Learning via Bilevel Optimization](https://mlsanity.com/paper/2409.05539/) (2024)
- [Clustered Federated Learning based on Nonconvex Pairwise Fusion](https://mlsanity.com/paper/2211.04218/) (2022)
- [Efficient Node Selection in Private Personalized Decentralized Learning](https://mlsanity.com/paper/2301.12755/) (2023)
- [Topology-Based Reconstruction Prevention for Decentralised Learning](https://mlsanity.com/paper/2312.05248/) (2023)
- [Low-Cost Privacy-Preserving Decentralized Learning](https://mlsanity.com/paper/2403.11795/) (2024)
- [Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environment](https://mlsanity.com/paper/2508.08278/) (2025)
- [A Generative Framework for Personalized Learning and Estimation: Theory, Algorithms, and Privacy](https://mlsanity.com/paper/2207.01771/) (2022)
- [Leveraging Federated Learning and Edge Computing for Recommendation Systems within Cloud Computing Networks](https://mlsanity.com/paper/2403.03165/) (2024)
- [Structured Cooperative Learning with Graphical Model Priors](https://mlsanity.com/paper/2306.09595/) (2023)
- [ADEPT: Hierarchical Bayes Approach to Personalized Federated Unsupervised Learning](https://mlsanity.com/paper/2402.12537/) (2024)
- [Communication-Efficient Personalized Distributed Learning with Data and Node Heterogeneity](https://mlsanity.com/paper/2504.17520/) (2025)
- [Federated Latent Class Regression for Hierarchical Data](https://mlsanity.com/paper/2206.10783/) (2022)
- [MAPL: Model Agnostic Peer-to-peer Learning](https://mlsanity.com/paper/2403.19792/) (2024)
- [P4: Towards private, personalized, and Peer-to-Peer learning](https://mlsanity.com/paper/2405.17697/) (2024)
- [Controlled privacy leakage propagation throughout overlapping grouped learning](https://mlsanity.com/paper/2503.04054/) (2025)
- [SPIRE: Conditional Personalization for Federated Diffusion Generative Models](https://mlsanity.com/paper/2506.12303/) (2025)
- [Client Clustering Meets Knowledge Sharing: Enhancing Privacy and Robustness in Personalized Peer-to-Peer Learning](https://mlsanity.com/paper/2506.20413/) (2025)

## Abstract (arXiv)

We consider the fully decentralized machine learning scenario where many users with personal datasets collaborate to learn models through local peer-to-peer exchanges, without a central coordinator. We propose to train personalized models that leverage a collaboration graph describing the relationships between user personal tasks, which we learn jointly with the models. Our fully decentralized optimization procedure alternates between training nonlinear models given the graph in a greedy boosting manner, and updating the collaboration graph (with controlled sparsity) given the models. Throughout the process, users exchange messages only with a small number of peers (their direct neighbors when updating the models, and a few random users when updating the graph), ensuring that the procedure naturally scales with the number of users. Overall, our approach is communication-efficient and avoids exchanging personal data. We provide an extensive analysis of the convergence rate, memory and communication complexity of our approach, and demonstrate its benefits compared to competing techniques on synthetic and real datasets.