# Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchange

arXiv: https://arxiv.org/abs/2208.02856 · HTML: https://mlsanity.com/paper/2208.02856/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 25 references that are in the map.
- First posted on arXiv 4 Aug 2022.

## Builds on (most cited first, 16 of 16)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Unsupervised Representation Learning](https://mlsanity.com/paper/2010.08982/) (2020)
- [Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregations](https://mlsanity.com/paper/2103.10481/) (2021)
- [Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks](https://mlsanity.com/paper/2007.09511/) (2020)
- [Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation](https://mlsanity.com/paper/2101.00787/) (2021)
- [Optimal Importance Sampling for Federated Learning](https://mlsanity.com/paper/2010.13600/) (2020)
- [Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clients](https://mlsanity.com/paper/2204.03304/) (2022)
- [Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks](https://mlsanity.com/paper/2202.02947/) (2022)

## Built on by (most cited first, 5 of 5)

- [Decentralized Federated Learning: A Survey and Perspective](https://mlsanity.com/paper/2306.01603/) (2023)
- [Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networks](https://mlsanity.com/paper/2311.04350/) (2023)
- [Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning](https://mlsanity.com/paper/2402.09629/) (2024)
- [Coded Matrix Computations for D2D-enabled Linearized Federated Learning](https://mlsanity.com/paper/2302.12305/) (2023)
- [Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labels](https://mlsanity.com/paper/2404.09861/) (2024)

## Abstract (arXiv)

Federated learning (FL) has been recognized as one of the most promising solutions for distributed machine learning (ML). In most of the current literature, FL has been studied for supervised ML tasks, in which edge devices collect labeled data. Nevertheless, in many applications, it is impractical to assume existence of labeled data across devices. To this end, we develop a novel methodology, Cooperative Federated unsupervised Contrastive Learning (CF-CL), for FL across edge devices with unlabeled datasets. CF-CL employs local device cooperation where data are exchanged among devices through device-to-device (D2D) communications to avoid local model bias resulting from non-independent and identically distributed (non-i.i.d.) local datasets. CF-CL introduces a push-pull smart data sharing mechanism tailored to unsupervised FL settings, in which, each device pushes a subset of its local datapoints to its neighbors as reserved data points, and pulls a set of datapoints from its neighbors, sampled through a probabilistic importance sampling technique. We demonstrate that CF-CL leads to (i) alignment of unsupervised learned latent spaces across devices, (ii) faster global convergence, allowing for less frequent global model aggregations; and (iii) is effective in extreme non-i.i.d. data settings across the devices.