# Decentralized adaptive clustering of deep nets is beneficial for client collaboration

arXiv: https://arxiv.org/abs/2206.08839 · HTML: https://mlsanity.com/paper/2206.08839/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 18 references that are in the map.
- First posted on arXiv 17 Jun 2022.

## Builds on (most cited first, 7 of 7)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Inverting Gradients -- How easy is it to break privacy in federated learning?](https://mlsanity.com/paper/2003.14053/) (2020)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [Gossip training for deep learning](https://mlsanity.com/paper/1611.09726/) (2016)
- [Decentralized federated learning of deep neural networks on non-iid data](https://mlsanity.com/paper/2107.08517/) (2021)

## Built on by (most cited first, 7 of 7)

- [Efficient Node Selection in Private Personalized Decentralized Learning](https://mlsanity.com/paper/2301.12755/) (2023)
- [Federated clustering with GAN-based data synthesis](https://mlsanity.com/paper/2210.16524/) (2022)
- [Towards Heterogeneity-Aware and Energy-Efficient Topology Optimization for Decentralized Federated Learning in Edge Environment](https://mlsanity.com/paper/2508.08278/) (2025)
- [On the effects of similarity metrics in decentralized deep learning under distributional shift](https://mlsanity.com/paper/2409.10720/) (2024)
- [Fair Decentralized Learning](https://mlsanity.com/paper/2410.02541/) (2024)
- [Concept-aware clustering for decentralized deep learning under temporal shift](https://mlsanity.com/paper/2306.12768/) (2023)
- [Client-Conditional Federated Learning via Local Training Data Statistics](https://mlsanity.com/paper/2603.11307/) (2026)

## Abstract (arXiv)

We study the problem of training personalized deep learning models in a decentralized peer-to-peer setting, focusing on the setting where data distributions differ between the clients and where different clients have different local learning tasks. We study both covariate and label shift, and our contribution is an algorithm which for each client finds beneficial collaborations based on a similarity estimate for the local task. Our method does not rely on hyperparameters which are hard to estimate, such as the number of client clusters, but rather continuously adapts to the network topology using soft cluster assignment based on a novel adaptive gossip algorithm. We test the proposed method in various settings where data is not independent and identically distributed among the clients. The experimental evaluation shows that the proposed method performs better than previous state-of-the-art algorithms for this problem setting, and handles situations well where previous methods fail.