# DELTA: Diverse Client Sampling for Fasting Federated Learning

arXiv: https://arxiv.org/abs/2205.13925 · HTML: https://mlsanity.com/paper/2205.13925/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 70 references that are in the map.
- First posted on arXiv 27 May 2022.

## Builds on (most cited first, 30 of 43)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Adaptive Federated Optimization](https://mlsanity.com/paper/2003.00295/) (2020)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [LEAF: A Benchmark for Federated Settings](https://mlsanity.com/paper/1812.01097/) (2018)
- [Ensemble Distillation for Robust Model Fusion in Federated Learning](https://mlsanity.com/paper/2006.07242/) (2020)
- [Personalized Federated Learning with Moreau Envelopes](https://mlsanity.com/paper/2006.08848/) (2020)
- [A Unified Theory of Decentralized SGD with Changing Topology and Local Updates](https://mlsanity.com/paper/2003.10422/) (2020)
- [HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients](https://mlsanity.com/paper/2010.01264/) (2020)
- [Not All Samples Are Created Equal: Deep Learning with Importance Sampling](https://mlsanity.com/paper/1803.00942/) (2018)
- [A Field Guide to Federated Optimization](https://mlsanity.com/paper/2107.06917/) (2021)
- [Parallel Restarted SGD with Faster Convergence and Less Communication: Demystifying Why Model Averaging Works for Deep Learning](https://mlsanity.com/paper/1807.06629/) (2018)
- [Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies](https://mlsanity.com/paper/2010.01243/) (2020)
- [Achieving Linear Speedup with Partial Worker Participation in Non-IID Federated Learning](https://mlsanity.com/paper/2101.11203/) (2021)
- [Better Theory for SGD in the Nonconvex World](https://mlsanity.com/paper/2002.03329/) (2020)
- [Mime: Mimicking Centralized Stochastic Algorithms in Federated Learning](https://mlsanity.com/paper/2008.03606/) (2020)
- [Optimal Client Sampling for Federated Learning](https://mlsanity.com/paper/2010.13723/) (2020)
- [Federated Graph Classification over Non-IID Graphs](https://mlsanity.com/paper/2106.13423/) (2021)
- [Clustered Sampling: Low-Variance and Improved Representativity for Clients Selection in Federated Learning](https://mlsanity.com/paper/2105.05883/) (2021)
- [Stochastic Optimization with Importance Sampling](https://mlsanity.com/paper/1401.2753/) (2014)
- [Variance Reduction in SGD by Distributed Importance Sampling](https://mlsanity.com/paper/1511.06481/) (2015)
- [Accelerating Federated Learning via Momentum Gradient Descent](https://mlsanity.com/paper/1910.03197/) (2019)
- [Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling](https://mlsanity.com/paper/2112.11256/) (2021)
- [Federated Multi-Armed Bandits](https://mlsanity.com/paper/2101.12204/) (2021)
- [Multi-Armed Bandit Based Client Scheduling for Federated Learning](https://mlsanity.com/paper/2007.02315/) (2020)
- [FedSoft: Soft Clustered Federated Learning with Proximal Local Updating](https://mlsanity.com/paper/2112.06053/) (2021)
- [A Unified Analysis of Federated Learning with Arbitrary Client Participation](https://mlsanity.com/paper/2205.13648/) (2022)
- [Communication-Efficient Federated Learning via Optimal Client Sampling](https://mlsanity.com/paper/2007.15197/) (2020)

## Built on by (most cited first, 24 of 24)

- [Non-IID data in Federated Learning: A Survey with Taxonomy, Metrics, Methods, Frameworks and Future Directions](https://mlsanity.com/paper/2411.12377/) (2024)
- [Channel and Gradient-Importance Aware Device Scheduling for Over-the-Air Federated Learning](https://mlsanity.com/paper/2305.16854/) (2023)
- [Exact and Linear Convergence for Federated Learning under Arbitrary Client Participation is Attainable](https://mlsanity.com/paper/2503.20117/) (2025)
- [Entropy-driven Fair and Effective Federated Learning](https://mlsanity.com/paper/2301.12407/) (2023)
- [Scalable Decentralized Learning with Teleportation](https://mlsanity.com/paper/2501.15259/) (2025)
- [CC-FedAvg: Computationally Customized Federated Averaging](https://mlsanity.com/paper/2212.13679/) (2022)
- [Enhanced Federated Optimization: Adaptive Unbiased Client Sampling with Reduced Variance](https://mlsanity.com/paper/2310.02698/) (2023)
- [Smart Sampling: Helping from Friendly Neighbors for Decentralized Federated Learning](https://mlsanity.com/paper/2407.04460/) (2024)
- [The Power of Bias: Optimizing Client Selection in Federated Learning with Heterogeneous Differential Privacy](https://mlsanity.com/paper/2408.08642/) (2024)
- [FedMABA: Towards Fair Federated Learning through Multi-Armed Bandits Allocation](https://mlsanity.com/paper/2410.20141/) (2024)
- [Energy-Efficient Federated Learning for AIoT using Clustering Methods](https://mlsanity.com/paper/2505.09704/) (2025)
- [Heterogeneity-Aware Client Sampling for Optimal and Efficient Federated Learning](https://mlsanity.com/paper/2505.11304/) (2025)
- [Optimal Client Sampling in Federated Learning with Client-Level Heterogeneous Differential Privacy](https://mlsanity.com/paper/2505.13655/) (2025)
- [FedRec+: Enhancing Privacy and Addressing Heterogeneity in Federated Recommendation Systems](https://mlsanity.com/paper/2310.20193/) (2023)
- [Client2Vec: Improving Federated Learning by Distribution Shifts Aware Client Indexing](https://mlsanity.com/paper/2405.16233/) (2024)
- [Unifying Physics- and Data-Driven Modeling via Novel Causal Spatiotemporal Graph Neural Network for Interpretable Epidemic Forecasting](https://mlsanity.com/paper/2504.05140/) (2025)
- [Overcoming Challenges of Partial Client Participation in Federated Learning : A Comprehensive Review](https://mlsanity.com/paper/2506.02887/) (2025)
- [SFedKD: Sequential Federated Learning with Discrepancy-Aware Multi-Teacher Knowledge Distillation](https://mlsanity.com/paper/2507.08508/) (2025)
- [HeteRo-Select: Informativeness as the Participation Driver in Heterogeneous Federated Learning](https://mlsanity.com/paper/2508.06692/) (2025)
- [SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning](https://mlsanity.com/paper/2510.23455/) (2025)
- [FedOAED: Federated On-Device Autoencoder Denoiser for Heterogeneous Data under Limited Client Availability](https://mlsanity.com/paper/2512.17986/) (2025)
- [FedAdaVR: Adaptive Variance Reduction for Robust Federated Learning under Limited Client Participation](https://mlsanity.com/paper/2601.22204/) (2026)
- [FedSteer: Taming Extreme Gradient Staleness in Federated Learning with Corrective Projections and Caching](https://mlsanity.com/paper/2606.10124/) (2026)
- [From Data Heterogeneity to Convergence: A Data-Centric Review of Federated Learning](https://mlsanity.com/paper/2606.10595/) (2026)

## Abstract (arXiv)

Partial client participation has been widely adopted in Federated Learning (FL) to reduce the communication burden efficiently. However, an inadequate client sampling scheme can lead to the selection of unrepresentative subsets, resulting in significant variance in model updates and slowed convergence. Existing sampling methods are either biased or can be further optimized for faster convergence.In this paper, we present DELTA, an unbiased sampling scheme designed to alleviate these issues. DELTA characterizes the effects of client diversity and local variance, and samples representative clients with valuable information for global model updates. In addition, DELTA is a proven optimal unbiased sampling scheme that minimizes variance caused by partial client participation and outperforms other unbiased sampling schemes in terms of convergence. Furthermore, to address full-client gradient dependence,we provide a practical version of DELTA depending on the available clients' information, and also analyze its convergence. Our results are validated through experiments on both synthetic and real-world datasets.