# Discovering Collaboration from Novelty: Random Network Distillation for Clustered Federated Learning

arXiv: https://arxiv.org/abs/2606.30499 · HTML: https://mlsanity.com/paper/2606.30499/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 17 references that are in the map.
- First posted on arXiv 29 Jun 2026.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Exploration by Random Network Distillation](https://mlsanity.com/paper/1810.12894/) (2018)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [An Efficient Framework for Clustered Federated Learning](https://mlsanity.com/paper/2006.04088/) (2020)
- [FBFL: A Field-Based Coordination Approach for Data Heterogeneity in Federated Learning](https://mlsanity.com/paper/2502.08577/) (2025)
- [ProFed: a Benchmark for Proximity-based non-IID Federated Learning](https://mlsanity.com/paper/2503.20618/) (2025)
- [Expert-Free Online Transfer Learning in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2303.01170/) (2023)
- [C2FL: Clustered Continual Federated Learning under Spatial and Temporal Drift](https://mlsanity.com/paper/2606.18003/) (2026)

## Abstract (arXiv)

Federated Learning often suffers under non-independently and identically distributed data, where a single global model may fail to represent the diversity of client distributions. Clustered Federated Learning mitigates this issue by training specialized models for groups of similar clients, but existing approaches often couple cluster assignment with the main training loop, increasing computational and communication costs. We propose a lightweight clustering approach based on Random Network Distillation. Each client trains a compact Random Network Distillation predictor on its local data and uses its prediction error as a novelty signal to estimate similarity with other clients. This enables the discovery of meaningful client groups before federated training, without sharing raw data or repeatedly evaluating the main model. Crucially, the resulting federations emerge from local novelty estimates at runtime, making the method suitable for autonomous large-scale distributed systems where neither the number of clusters nor the collaboration structure can be specified a priori. Overall, by decoupling clustering from learning, the method provides a task-agnostic and efficient mechanism for autonomous collaboration under non-independently and identically distributed data.