mlsanityData to 7 Oct 2026

Device Sampling for Heterogeneous Federated Learning: Theory, Algorithms, and Implementation

arXiv 2101.00787 ↗First posted 4 Jan 2021cs.NI cs.DC cs.LGcited by 40

Not on the map. Why?

Abstract (arXiv)

The conventional federated learning (FedL) architecture distributes machine learning (ML) across worker devices by having them train local models that are periodically aggregated by a server. FedL ignores two important characteristics of contemporary wireless networks, however: (i) the network may contain heterogeneous communication/computation resources, while (ii) there may be significant overlaps in devices' local data distributions. In this work, we develop a novel optimization methodology that jointly accounts for these factors via intelligent device sampling complemented by device-to-device (D2D) offloading. Our optimization aims to select the best combination of sampled nodes and data offloading configuration to maximize FedL training accuracy subject to realistic constraints on the network topology and device capabilities. Theoretical analysis of the D2D offloading subproblem leads to new FedL convergence bounds and an efficient sequential convex optimizer. Using this result, we develop a sampling methodology based on graph convolutional networks (GCNs) which learns the relationship between network attributes, sampled nodes, and resulting offloading that maximizes FedL accuracy. Through evaluation on real-world datasets and network measurements from our IoT testbed, we find that our methodology while sampling less than 5% of all devices outperforms conventional FedL substantially both in terms of trained model accuracy and required resource utilization.

Key facts. Cited by 40 papers in the map (data through 7 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 4 Jan 2021.

Builds on

29 of 49 references.

References · 29

  1. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2019Advances and Open Problems in Federated Learningcited by 3,021
  4. 2019A Comprehensive Survey on Graph Neural Networkscited by 2,906
  5. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  6. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  7. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  8. 2019FedMD: Heterogenous Federated Learning via Model Distillationcited by 454
  9. 2019Agnostic Federated Learningcited by 451
  10. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  11. 2018Adaptive Federated Learning in Resource Constrained Edge Computing Systemscited by 408
  12. 2019Fair Resource Allocation in Federated Learningcited by 389
17 more
  1. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
  2. 2018Combinatorial Optimization with Graph Convolutional Networks and Guided Tree Searchcited by 182
  3. 2019Energy Efficient Federated Learning Over Wireless Communication Networkscited by 181
  4. 2019Scheduling Policies for Federated Learning in Wireless Networkscited by 131
  5. 2019Federated Learning for Wireless Communications: Motivation, Opportunities and Challengescited by 128
  6. 2019Fairness risk measurescited by 85
  7. 2020Scheduling for Cellular Federated Edge Learning with Importance and Channel Awarenesscited by 62
  8. 2020Fast-Convergent Federated Learningcited by 61
  9. 2017Deep Learning for IoT Big Data and Streaming Analytics: A Surveycited by 59
  10. 2020Multi-Armed Bandit Based Client Scheduling for Federated Learningcited by 56
  11. 2019Device Scheduling with Fast Convergence for Wireless Federated Learningcited by 46
  12. 2020Multi-Stage Hybrid Federated Learning over Large-Scale D2D-Enabled Fog Networkscited by 42
  13. 2020Network-Aware Optimization of Distributed Learning for Fog Computingcited by 27
  14. 2020Dynamic Sampling and Selective Masking for Communication-Efficient Federated Learningcited by 16
  15. 2018Secure Federated Transfer Learningcited by 13
  16. 2020A Fast Graph Neural Network-Based Method for Winner Determination in Multi-Unit Combinatorial Auctionscited by 6
  17. 2020Joint Optimization of Signal Design and Resource Allocation in Wireless D2D Edge Computingcited by 2

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 12; 2023: 16; 2024: 7.

40 papers.

Sort:

By area:

  1. 2022The Right to be Forgotten in Federated Learning: An Efficient Realization with Rapid Retrainingcited by 109
  2. 2023Decentralized Federated Learning: A Survey and Perspectivecited by 86
  3. 2021Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Samplingcited by 66
  4. 2021Semi-Decentralized Federated Learning with Cooperative D2D Local Model Aggregationscited by 50
  5. 2024Automated Federated Pipeline for Parameter-Efficient Fine-Tuning of Large Language Modelscited by 49
  6. 2023FwdLLM: Efficient FedLLM using Forward Gradientcited by 25
  7. 2022FedAdapter: Efficient Federated Learning for Modern NLPcited by 24
  8. 2023Green Edge AI: A Contemporary Surveycited by 24
  9. 2023Topology-aware Federated Learning in Edge Computing: A Comprehensive Surveycited by 23
  10. 2021UAV-assisted Online Machine Learning over Multi-Tiered Networks: A Hierarchical Nested Personalized Federated Learning Approachcited by 20
  11. 2022SplitGP: Achieving Both Generalization and Personalization in Federated Learningcited by 20
  12. 2022Parallel Successive Learning for Dynamic Distributed Model Training over Heterogeneous Wireless Networkscited by 18
28 more
  1. 2022Federated Few-Shot Learning for Mobile NLPcited by 15
  2. 2021Enabling Long-Term Cooperation in Cross-Silo Federated Learning: A Repeated Game Perspectivecited by 14
  3. 2022Training Mixed-Domain Translation Models via Federated Learningcited by 12
  4. 2024Advances in Robust Federated Learning: A Survey with Heterogeneity Considerationscited by 12
  5. 2022A Survey on Participant Selection for Federated Learning in Mobile Networkscited by 11
  6. 2022Multi-Edge Server-Assisted Dynamic Federated Learning with an Optimized Floating Aggregation Pointcited by 10
  7. 2023AnycostFL: Efficient On-Demand Federated Learning over Heterogeneous Edge Devicescited by 10
  8. 2023GA-DRL: Graph Neural Network-Augmented Deep Reinforcement Learning for DAG Task Scheduling over Dynamic Vehicular Cloudscited by 10
  9. 2023Privacy-Preserving Graph Machine Learning from Data to Computation: A Surveycited by 10
  10. 2023Delay-Aware Hierarchical Federated Learningcited by 9
  11. 2024Adaptive Heterogeneous Client Sampling for Federated Learning over Wireless Networkscited by 9
  12. 2023Multi-Source to Multi-Target Decentralized Federated Domain Adaptationcited by 6
  13. 2021Communication-Efficient Federated Learning for Neural Machine Translationcited by 5
  14. 2022Embedding Alignment for Unsupervised Federated Learning via Smart Data Exchangecited by 5
  15. 2022Boost Decentralized Federated Learning in Vehicular Networks by Diversifying Data Sourcescited by 5
  16. 2022Towards Practical Few-shot Federated NLPcited by 5
  17. 2023Truthful Incentive Mechanism for Federated Learning with Crowdsourced Data Labelingcited by 4
  18. 2023Trustworthy Federated Learning: A Surveycited by 4
  19. 2023Digital Twin-Assisted Knowledge Distillation Framework for Heterogeneous Federated Learningcited by 3
  20. 2023Asynchronous Multi-Model Dynamic Federated Learning over Wireless Networks: Theory, Modeling, and Optimizationcited by 3
  21. 2024Decentralized Federated Learning with Gradient Tracking over Time-Varying Directed Networkscited by 3
  22. 2022Contextual Model Aggregation for Fast and Robust Federated Learning in Edge Computingcited by 2
  23. 2023FedAVO: Improving Communication Efficiency in Federated Learning with African Vultures Optimizercited by 2
  24. 2023Device Sampling and Resource Optimization for Federated Learning in Cooperative Edge Networkscited by 2
  25. 2024Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learningcited by 2
  26. 2023Coded Matrix Computations for D2D-enabled Linearized Federated Learningcited by 0
  27. 2024Unsupervised Federated Optimization at the Edge: D2D-Enabled Learning without Labelscited by 0
  28. 2024FedEx: Expediting Federated Learning over Heterogeneous Mobile Devices by Overlapping and Participant Selectioncited by 0