mlsanityData to 14 Oct 2026

On the effects of similarity metrics in decentralized deep learning under distributional shift

arXiv 2409.10720 ↗First posted 16 Sep 2024cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Decentralized Learning (DL) enables privacy-preserving collaboration among organizations or users to enhance the performance of local deep learning models. However, model aggregation becomes challenging when client data is heterogeneous, and identifying compatible collaborators without direct data exchange remains a pressing issue. In this paper, we investigate the effectiveness of various similarity metrics in DL for identifying peers for model merging, conducting an empirical analysis across multiple datasets with distribution shifts. Our research provides insights into the performance of these metrics, examining their role in facilitating effective collaboration. By exploring the strengths and limitations of these metrics, we contribute to the development of robust DL methods.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 14 of its 27 references that are in the map. First posted on arXiv 16 Sep 2024.

Builds on

14 of 27 references.

References · 14

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,349
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2021Model-Contrastive Federated Learningcited by 766
  6. 2020An Efficient Framework for Clustered Federated Learningcited by 511
  7. 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 503
  8. 2019Robust Federated Learning in a Heterogeneous Environmentcited by 90
  9. 2022Where to Begin? On the Impact of Pre-Training and Initialization in Federated Learningcited by 71
  10. 2016Gossip training for deep learningcited by 35
  11. 2021Decentralized federated learning of deep neural networks on non-iid datacited by 25
  12. 2022Towards Effective Clustered Federated Learning: A Peer-to-peer Framework with Adaptive Neighbor Matchingcited by 24
2 more
  1. 2022Decentralized adaptive clustering of deep nets is beneficial for client collaborationcited by 7
  2. 2023Efficient Node Selection in Private Personalized Decentralized Learningcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Chisme: Heterogeneity-Aware Gossip Learningcited by 0