mlsanityData to 14 Oct 2026

DELTA: Diverse Client Sampling for Fasting Federated Learning

arXiv 2205.13925 ↗First posted 27 May 2022cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Partial client participation has been widely adopted in Federated Learning (FL) to reduce the communication burden efficiently. However, an inadequate client sampling scheme can lead to the selection of unrepresentative subsets, resulting in significant variance in model updates and slowed convergence. Existing sampling methods are either biased or can be further optimized for faster convergence.In this paper, we present DELTA, an unbiased sampling scheme designed to alleviate these issues. DELTA characterizes the effects of client diversity and local variance, and samples representative clients with valuable information for global model updates. In addition, DELTA is a proven optimal unbiased sampling scheme that minimizes variance caused by partial client participation and outperforms other unbiased sampling schemes in terms of convergence. Furthermore, to address full-client gradient dependence,we provide a practical version of DELTA depending on the available clients' information, and also analyze its convergence. Our results are validated through experiments on both synthetic and real-world datasets.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Builds on 43 of its 70 references that are in the map. First posted on arXiv 27 May 2022.

Builds on

43 of 70 references.

References · 43

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  2. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  3. 2019On the Convergence of FedAvg on Non-IID Datacited by 1,229
  4. 2020Adaptive Federated Optimizationcited by 1,046
  5. 2020Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimizationcited by 906
  6. 2019Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classificationcited by 851
  7. 2018LEAF: A Benchmark for Federated Settingscited by 792
  8. 2020Ensemble Distillation for Robust Model Fusion in Federated Learningcited by 682
  9. 2020Personalized Federated Learning with Moreau Envelopescited by 618
  10. 2020A Unified Theory of Decentralized SGD with Changing Topology and Local Updatescited by 367
  11. 2020HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clientscited by 337
  12. 2018Not All Samples Are Created Equal: Deep Learning with Importance Samplingcited by 328
31 more
  1. 2021A Field Guide to Federated Optimizationcited by 315
  2. 2018Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learningcited by 278
  3. 2020Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategiescited by 198
  4. 2021Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learningcited by 193
  5. 2020Better Theory for SGD in the Nonconvex Worldcited by 158
  6. 2020Mime: Mimicking Centralized Stochastic Algorithms in Federated Learningcited by 154
  7. 2020Optimal Client Sampling for Federated Learningcited by 124
  8. 2021Federated Graph Classification over Non-IID Graphscited by 114
  9. 2021Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learningcited by 93
  10. 2014Stochastic Optimization with Importance Samplingcited by 91
  11. 2015Variance Reduction in SGD by Distributed Importance Samplingcited by 79
  12. 2019Accelerating Federated Learning via Momentum Gradient Descentcited by 76
  13. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  14. 2021Federated Multi-Armed Banditscited by 59
  15. 2020Multi-Armed Bandit Based Client Scheduling for Federated Learningcited by 57
  16. 2021FedSoft: Soft Clustered Federated Learning with Proximal Local Updatingcited by 46
  17. 2022A Unified Analysis of Federated Learning with Arbitrary Client Participationcited by 45
  18. 2020Communication-Efficient Federated Learning via Optimal Client Samplingcited by 39
  19. 2020Stochastic Client Selection for Federated Learning with Volatile Clientscited by 39
  20. 2017Biased Importance Sampling for Deep Neural Network Trainingcited by 31
  21. 2020Bandit-based Communication-Efficient Client Selection Strategies for Federated Learningcited by 29
  22. 2022Communication-Efficient Device Scheduling for Federated Learning Using Stochastic Optimizationcited by 28
  23. 2020Dynamic Clustering in Federated Learningcited by 27
  24. 2020Federated Learning under Importance Samplingcited by 26
  25. 2021Towards Federated Learning on Time-Evolving Heterogeneous Datacited by 23
  26. 2021Context-Aware Online Client Selection for Hierarchical Federated Learningcited by 20
  27. 2017Safe Adaptive Importance Samplingcited by 19
  28. 2020Budgeted Online Selection of Candidate IoT Clients to Participate in Federated Learningcited by 14
  29. 2021A General Theory for Client Sampling in Federated Learningcited by 8
  30. 2021Adaptive Client Sampling in Federated Learning via Online Learning with Bandit Feedbackcited by 7
  31. 2022Coresets for Weight-Constrained Anisotropic Assignment and Clusteringcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 4; 2024: 5; 2025: 11; 2026: 3.

24 papers.

Sort:

By area:

  1. 2024Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directionscited by 13
  2. 2023Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learningcited by 9
  3. 2025Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainablecited by 6
  4. 2023Entropy-driven Fair and Effective Federated Learningcited by 4
  5. 2025Scalable Decentralized Learning with Teleportationcited by 3
  6. 2022CC-FedAvg: Computationally Customized Federated Averagingcited by 2
  7. 2023Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variancecited by 2
  8. 2024Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learningcited by 2
  9. 2024The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacycited by 2
  10. 2024FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocationcited by 1
  11. 2025Energy-Efficient Federated Learning for AIoT using Clustering Methodscited by 1
  12. 2025Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learningcited by 1
12 more
  1. 2025Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacycited by 1
  2. 2023FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systemscited by 0
  3. 2024Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexingcited by 0
  4. 2025Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecastingcited by 0
  5. 2025Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Reviewcited by 0
  6. 2025SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillationcited by 0
  7. 2025HeteRo-Select: Informativeness as the Participation Driver in Heterogeneous Federated Learningcited by 0
  8. 2025SGFusion: Stochastic Geographic Gradient Fusion in Federated Learningcited by 0
  9. 2025FedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availabilitycited by 0
  10. 2026FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participationcited by 0
  11. 2026FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Cachingcited by 0
  12. 2026From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learningcited by 0