# Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labels

arXiv: https://arxiv.org/abs/2404.09861 · HTML: https://mlsanity.com/paper/2404.09861/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Federated Learning: Challenges, Methods, and Future Directions](https://mlsanity.com/paper/1908.07873/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Model-Contrastive Federated Learning](https://mlsanity.com/paper/2103.16257/) (2021)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [FedMD: Heterogenous Federated Learning via Model Distillation](https://mlsanity.com/paper/1910.03581/) (2019)
- [Adaptive Federated Learning in Resource Constrained Edge Computing Systems](https://mlsanity.com/paper/1804.05271/) (2018)
- [Federated learning with hierarchical clustering of local updates to improve training on non-IID data](https://mlsanity.com/paper/2004.11791/) (2020)
- [Federated Unsupervised Representation Learning](https://mlsanity.com/paper/2010.08982/) (2020)
- [Stochastic Optimization with Importance Sampling](https://mlsanity.com/paper/1401.2753/) (2014)
- [Divergence-aware Federated Self-Supervised Learning](https://mlsanity.com/paper/2204.04385/) (2022)
- [Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregations](https://mlsanity.com/paper/2103.10481/) (2021)
- [Federated Learning Meets Natural Language Processing: A Survey](https://mlsanity.com/paper/2107.12603/) (2021)
- [Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networks](https://mlsanity.com/paper/2007.09511/) (2020)
- [Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation](https://mlsanity.com/paper/2101.00787/) (2021)
- [Importance Sampling for Minibatches](https://mlsanity.com/paper/1602.02283/) (2016)
- [Federated User Representation Learning](https://mlsanity.com/paper/1909.12535/) (2019)
- [Federated Learning for Privacy Preservation in Smart Healthcare Systems: A Comprehensive Survey](https://mlsanity.com/paper/2203.09702/) (2022)
- [Optimal Importance Sampling for Federated Learning](https://mlsanity.com/paper/2010.13600/) (2020)
- [Federated Learning from Only Unlabeled Data with Class-Conditional-Sharing Clients](https://mlsanity.com/paper/2204.03304/) (2022)
- [FedDis: Disentangled Federated Learning for Unsupervised Brain Pathology Segmentation](https://mlsanity.com/paper/2103.03705/) (2021)
- [Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networks](https://mlsanity.com/paper/2202.02947/) (2022)
- [Learning-Based Computation Offloading for IoT Devices with Energy Harvesting](https://mlsanity.com/paper/1712.08768/) (2017)
- [Federated Few-Shot Learning with Adversarial Learning](https://mlsanity.com/paper/2104.00365/) (2021)
- [FedCon: A Contrastive Framework for Federated Semi-Supervised Learning](https://mlsanity.com/paper/2109.04533/) (2021)
- [Towards Cooperative Federated Learning over Heterogeneous Edge/Fog Networks](https://mlsanity.com/paper/2303.08361/) (2023)
- [Can we Generalize and Distribute Private Representation Learning?](https://mlsanity.com/paper/2010.01792/) (2020)

## Abstract (arXiv)

Federated learning (FL) is a popular solution for distributed machine learning (ML). While FL has traditionally been studied for supervised ML tasks, in many applications, it is impractical to assume availability of labeled data across devices. To this end, we develop Cooperative Federated unsupervised Contrastive Learning (tt CF-CL) to facilitate FL across edge devices with unlabeled datasets. tt CF-CL employs local device cooperation where either explicit (i.e., raw data) or implicit (i.e., embeddings) information is exchanged through device-to-device (D2D) communications to improve local diversity. Specifically, we introduce a textitsmart information push-pull methodology for data/embedding exchange tailored to FL settings with either soft or strict data privacy restrictions. Information sharing is conducted through a probabilistic importance sampling technique at receivers leveraging a carefully crafted reserve dataset provided by transmitters. In the implicit case, embedding exchange is further integrated into the local ML training at the devices via a regularization term incorporated into the contrastive loss, augmented with a dynamic contrastive margin to adjust the volume of latent space explored. Numerical evaluations demonstrate that tt CF-CL leads to alignment of latent spaces learned across devices, results in faster and more efficient global model training, and is effective in extreme non-i.i.d. data distribution settings across devices.