# Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning

arXiv: https://arxiv.org/abs/2402.09629 · HTML: https://mlsanity.com/paper/2402.09629/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 11 of its 19 references that are in the map.
- First posted on arXiv 15 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 11 of 11)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [TernGrad: Ternary Gradients to Reduce Communication in Distributed Deep Learning](https://mlsanity.com/paper/1705.07878/) (2017)
- [Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation](https://mlsanity.com/paper/2101.00787/) (2021)
- [Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchange](https://mlsanity.com/paper/2208.02856/) (2022)
- [An unsupervised deep learning framework for medical image denoising](https://mlsanity.com/paper/2103.06575/) (2021)
- [Computation Offloading with Multiple Agents in Edge-Computing-Supported IoT](https://mlsanity.com/paper/2308.00463/) (2023)
- [A Reinforcement Learning-Based Approach to Graph Discovery in D2D-Enabled Federated Learning](https://mlsanity.com/paper/2308.03933/) (2023)

## Built on by (most cited first, 2 of 2)

- [Cooperative Decentralized Backdoor Attacks on Vertical Federated Learning](https://mlsanity.com/paper/2501.09320/) (2025)
- [TAP: Two-Stage Adaptive Personalization of Multi-Task and Multi-Modal Foundation Models in Federated Learning](https://mlsanity.com/paper/2509.26524/) (2025)

## Abstract (arXiv)

One of the main challenges of decentralized machine learning paradigms such as Federated Learning (FL) is the presence of local non-i.i.d. datasets. Device-to-device transfers (D2D) between distributed devices has been shown to be an effective tool for dealing with this problem and robust to stragglers. In an unsupervised case, however, it is not obvious how data exchanges should take place due to the absence of labels. In this paper, we propose an approach to create an optimal graph for data transfer using Reinforcement Learning. The goal is to form links that will provide the most benefit considering the environment's constraints and improve convergence speed in an unsupervised FL environment. Numerical analysis shows the advantages in terms of convergence speed and straggler resilience of the proposed method to different available FL schemes and benchmark datasets.