Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning
Not on the map. Why?
Abstract (arXiv)
This work addresses the problem of optimizing communications between server and clients in federated learning (FL). Current sampling approaches in FL are either biased, or non optimal in terms of server-clients communications and training stability. To overcome this issue, we introduce textitclustered sampling for clients selection. We prove that clustered sampling leads to better clients representatitivity and to reduced variance of the clients stochastic aggregation weights in FL. Compatibly with our theory, we provide two different clustering approaches enabling clients aggregation based on 1) sample size, and 2) models similarity. Through a series of experiments in non-iid and unbalanced scenarios, we demonstrate that model aggregation through clustered sampling consistently leads to better training convergence and variability when compared to standard sampling approaches. Our approach does not require any additional operation on the clients side, and can be seamlessly integrated in standard FL implementations. Finally, clustered sampling is compatible with existing methods and technologies for privacy enhancement, and for communication reduction through model compression.
Key facts. Cited by 93 papers in the map (data through 7 Oct 2026). Builds on 16 of its 23 references that are in the map. First posted on arXiv 12 May 2021.
Builds on
16 of 23 references.
References · 16
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2018Federated Optimization in Heterogeneous Networkscited by 2,971
- 2019On the Convergence of FedAvg on Non-IID Datacited by 1,220
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 897
- 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 836
- 2018Local SGD Converges Fast and Communicates Littlecited by 595
- 2019Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraintscited by 499
- 2018Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edgecited by 347
- 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 277
- 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
- 2018Don't Use Large Mini-Batches, Use Local SGDcited by 201
4 more
- 2020Is Local SGD Better than Minibatch SGD?cited by 151
- 2020Optimal Client Sampling for Federated Learningcited by 123
- 2019Local SGD with Periodic Averaging: Tighter Analysis and Adaptive Synchronizationcited by 106
- 2019MATCHA: Speeding Up Decentralized SGD via Matching Decomposition Samplingcited by 67
Built on by
Citing papers per year: 2021: 3; 2022: 19; 2023: 25; 2024: 22; 2025: 19; 2026: 5.
93 papers.
Sort:
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
- 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
- 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
- 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
- 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
- 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
- 2021Context-Aware Online Client Selection for Hierarchical Federated Learningcited by 20
- 2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
- 2023Federated Learning under Heterogeneous and Correlated Client Availabilitycited by 18
- 2023Federated Learning with Regularized Client Participationcited by 13
38 more
- 2023Is Aggregation the Only Choice? Federated Learning via Layer-wise Model Recombinationcited by 13
- 2022Federated Learning in Non-IID Settings Aided by Differentially Private Synthetic Datacited by 12
- 2022Federated Learning with Flexible Controlcited by 12
- 2022SAFARI: Sparsity enabled Federated Learning with Limited and Unreliable Communicationscited by 11
- 2022A Survey on Participant Selection for Federated Learning in Mobile Networkscited by 11
- 2022FedGS: Federated Graph-based Sampling with Arbitrary Client Availabilitycited by 10
- 2024Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networkscited by 9
- 2021From Deterioration to Acceleration: A Calibration Approach to Rehabilitating Step Asynchronism in Federated Optimizationcited by 8
- 2022Fast Heterogeneous Federated Learning with Hybrid Client Selectioncited by 8
- 2023FedSampling: A Better Sampling Strategy for Federated Learningcited by 8
- 2023Federated Learning While Providing Model as a Service: Joint Training and Inference Optimizationcited by 8
- 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysiscited by 8
- 2022GlueFL: Reconciling Client Sampling and Model Masking for Bandwidth Efficient Federated Learningcited by 7
- 2023Differentially Private Federated Clustering over Non-IID Datacited by 7
- 2023Improving Accelerated Federated Learning with Compression and Importance Samplingcited by 7
- 2024FedStale: leveraging stale client updates in federated learningcited by 5
- 2021Federated Submodel Optimization for Hot and Cold Data Featurescited by 4
- 2023Federated Learning on Non-iid Data via Local and Global Distillationcited by 4
- 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspectivecited by 4
- 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learningcited by 4
- 2022Variance-Reduced Heterogeneous Federated Learning via Stratified Client Selectioncited by 3
- 2022FedEntropy: Efficient Device Grouping for Federated Learning Using Maximum Entropy Judgmentcited by 3
- 2023Stabilizing and Improving Federated Learning with Non-IID Data and Client Dropoutcited by 3
- 2023DPP-based Client Selection for Federated Learning with Non-IID Datacited by 3
- 2024Review of Mathematical Optimization in Federated Learningcited by 3
- 2025Scalable Decentralized Learning with Teleportationcited by 3
- 2025FedSC: Federated Learning with Semantic-Aware Collaborationcited by 3
- 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participationcited by 3
- 2022CC-FedAvg: Computationally Customized Federated Averagingcited by 2
- 2023FLIPS: Federated Learning using Intelligent Participant Selectioncited by 2
- 2023Heterogeneity-Guided Client Sampling: Towards Fast and Efficient Non-IID Federated Learningcited by 2
- 2023Multi-Criteria Client Selection and Scheduling with Fairness Guarantee for Federated Learning Servicecited by 2
- 2024Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalizationcited by 2
- 2025Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devicescited by 2
- 2022FedCAT: Towards Accurate Federated Learning via Device Concatenationcited by 1
- 2023A Lightweight Method for Tackling Unknown Participation Statistics in Federated Averagingcited by 1
- 2023Structured Cooperative Learning with Graphical Model Priorscited by 1
- 2023Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variancecited by 1
50 most cited of 93 All 93 (JSON)
- 2026Cluster-Aware Over-the-Air Federated Learning with Energy-Harvesting Devices: From Global Training to Model Personalization2 Aug 2026
- 2026FedSurrogate: Backdoor Defense in Federated Learning via Layer Criticality and Surrogate Replacement11 May 2026
- 2026DeRelayL: Sustainable Decentralized Relay Learning30 Apr 2026
- 2026Personalized Federated Learning via Gaussian Generative Modeling12 Mar 2026
- 2026Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning24 Feb 2026
- 2025Sharpness-aware Federated Graph Learning18 Dec 2025
- 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning23 Oct 2025
- 2025CEPerFed: Communication-Efficient Personalized Federated Learning for Multi-Pulse MRI Classification20 Oct 2025
- 2025FedSSG: Expectation-Gated and History-Aware Drift Alignment for Federated Learning17 Sep 2025
- 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation3 Sep 2025
- 2025FedSC: Federated Learning with Semantic-Aware Collaboration26 Jun 2025
- 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review3 Jun 2025
38 more
- 2025Addressing Data Quality Decompensation in Federated Learning via Dynamic Client Selection27 May 2025
- 2025Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning16 May 2025
- 2025Energy-Efficient Federated Learning for AIoT using Clustering Methods14 May 2025
- 2025Balancing Client Participation in Federated Learning Using AoI8 May 2025
- 2025Stratify: Rethinking Federated Learning for Non-IID Data through Balanced Sampling18 Apr 2025
- 2025Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecasting7 Apr 2025
- 2025Secure Generalization through Stochastic Bidirectional Parameter Updates Using Dual-Gradient Mechanism3 Apr 2025
- 2025Client Selection in Federated Learning with Data Heterogeneity and Network Latencies2 Apr 2025
- 2025PAUSE: Low-Latency and Privacy-Aware Active User Selection for Federated Learning17 Mar 2025
- 2025Communication-Efficient Device Scheduling for Federated Learning Using Lyapunov Optimization1 Mar 2025
- 2025Update Estimation and Scheduling for Over-the-Air Federated Learning with Energy Harvesting Devices30 Jan 2025
- 2025Scalable Decentralized Learning with Teleportation25 Jan 2025
- 2024FedSTaS: Client Stratification and Client Level Sampling for Efficient Federated Learning18 Dec 2024
- 2024Review of Mathematical Optimization in Federated Learning2 Dec 2024
- 2024FedQP: Towards Accurate Federated Learning using Quadratic Programming Guided Mutation24 Nov 2024
- 2024FedSub: Introducing Class-aware Subnetworks Fusion to Enhance Personalized Federated Learning13 Nov 2024
- 2024Federated Learning under Periodic Client Participation and Heterogeneous Data: A New Communication-Efficient Algorithm and Analysis30 Oct 2024
- 2024Equitable Federated Learning with Activation Clustering24 Oct 2024
- 2024Debiasing Federated Learning with Correlated Client Participation2 Oct 2024
- 2024TPFL: Tsetlin-Personalized Federated Learning with Confidence-Based Clustering16 Sep 2024
- 2024Addressing Skewed Heterogeneity via Federated Prototype Rectification with Personalization15 Aug 2024
- 2024Load Balancing in Federated Learning1 Aug 2024
- 2024FedAR: Addressing Client Unavailability in Federated Learning with Local Update Approximation and Rectification26 Jul 2024
- 2024FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selection1 Jul 2024
- 2024Efficient Data Distribution Estimation for Accelerated Federated Learning3 Jun 2024
- 2024Exploring the Practicality of Federated Learning: A Survey Towards the Communication Perspective30 May 2024
- 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexing25 May 2024
- 2024FedStale: leveraging stale client updates in federated learning7 May 2024
- 2024Towards Fairness in Provably Communication-Efficient Federated Recommender Systems3 May 2024
- 2024Federated Learning with Only Positive Labels by Exploring Label Correlations24 Apr 2024
- 2024Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networks22 Apr 2024
- 2024DA-PFL: Dynamic Affinity Aggregation for Personalized Federated Learning14 Mar 2024
- 2024Decoupled Federated Learning on Long-Tailed and Non-IID data with Feature Statistics13 Mar 2024
- 2024Federated Learning Can Find Friends That Are Advantageous7 Feb 2024
- 2023LEFL: Low Entropy Client Sampling in Federated Learning29 Dec 2023
- 2023Harnessing the Power of Federated Learning in Federated Contextual Bandits26 Dec 2023
- 2023Federated Learning While Providing Model as a Service: Joint Training and Inference Optimization20 Dec 2023
- 2023Multi-Criteria Client Selection and Scheduling with Fairness Guarantee for Federated Learning Service5 Dec 2023
50 newest of 93 All 93 (JSON)
- 2022Fine-tuning Global Model via Data-Free Knowledge Distillation for Non-IID Federated Learningcited by 172
- 2023Personalized Federated Learning with Feature Alignment and Classifier Collaborationcited by 97
- 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
- 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
- 2022Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reductioncited by 32
- 2022A Multi-agent Reinforcement Learning Approach for Efficient Client Selection in Federated Learningcited by 28
- 2022DELTA: Diverse Client Sampling for Fasting Federated Learningcited by 24
- 2022FedCross: Towards Accurate Federated Learning via Multi-Model Cross-Aggregationcited by 21
- 2021Context-Aware Online Client Selection for Hierarchical Federated Learningcited by 20
- 2023MP-FedCL: Multiprototype Federated Contrastive Learning for Edge Intelligencecited by 20
- 2023Federated Learning under Heterogeneous and Correlated Client Availabilitycited by 18
- 2023Federated Learning with Regularized Client Participationcited by 13
12 most cited of 93 All 93 (JSON)
- 2026Cluster-Aware Over-the-Air Federated Learning with Energy-Harvesting Devices: From Global Training to Model Personalization2 Aug 2026
- 2026FedSurrogate: Backdoor Defense in Federated Learning via Layer Criticality and Surrogate Replacement11 May 2026
- 2026DeRelayL: Sustainable Decentralized Relay Learning30 Apr 2026
- 2026Personalized Federated Learning via Gaussian Generative Modeling12 Mar 2026
- 2026Heterogeneity-Aware Client Selection Methodology For Efficient Federated Learning24 Feb 2026
- 2025Sharpness-aware Federated Graph Learning18 Dec 2025
- 2025FedGPS: Statistical Rectification Against Data Heterogeneity in Federated Learning23 Oct 2025
- 2025CEPerFed: Communication-Efficient Personalized Federated Learning for Multi-Pulse MRI Classification20 Oct 2025
- 2025FedSSG: Expectation-Gated and History-Aware Drift Alignment for Federated Learning17 Sep 2025
- 2025Delayed Momentum Aggregation: Communication-efficient Byzantine-robust Federated Learning with Partial Participation3 Sep 2025
- 2025FedSC: Federated Learning with Semantic-Aware Collaboration26 Jun 2025
- 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review3 Jun 2025
12 newest of 93 All 93 (JSON)