mlsanityData to 7 Oct 2026

Resource-constrained Federated Edge Learning with Heterogeneous Data: Formulation and Analysis

arXiv 2110.07567 ↗First posted 14 Oct 2021cs.LG cs.CR cs.NIcited by 10

Not on the map. Why?

Abstract (arXiv)

Efficient collaboration between collaborative machine learning and wireless communication technology, forming a Federated Edge Learning (FEEL), has spawned a series of next-generation intelligent applications. However, due to the openness of network connections, the FEEL framework generally involves hundreds of remote devices (or clients), resulting in expensive communication costs, which is not friendly to resource-constrained FEEL. To address this issue, we propose a distributed approximate Newton-type algorithm with fast convergence speed to alleviate the problem of FEEL resource (in terms of communication resources) constraints. Specifically, the proposed algorithm is improved based on distributed L-BFGS algorithm and allows each client to approximate the high-cost Hessian matrix by computing the low-cost Fisher matrix in a distributed manner to find a "better" descent direction, thereby speeding up convergence. Second, we prove that the proposed algorithm has linear convergence in strongly convex and non-convex cases and analyze its computational and communication complexity. Similarly, due to the heterogeneity of the connected remote devices, FEEL faces the challenge of heterogeneous data and non-IID (Independent and Identically Distributed) data. To this end, we design a simple but elegant training scheme, namely FedOVA, to solve the heterogeneous statistical challenge brought by heterogeneous data. In this way, FedOVA first decomposes a multi-class classification problem into more straightforward binary classification problems and then combines their respective outputs using ensemble learning. In particular, the scheme can be well integrated with our communication efficient algorithm to serve FEEL. Numerical results verify the effectiveness and superiority of the proposed algorithm.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 33 of its 61 references that are in the map. First posted on arXiv 14 Oct 2021.

Builds on

33 of 61 references.

References · 33

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  3. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  4. 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
  5. 2018Federated Learning with Non-IID Datacited by 1,099
  6. 2019Deep Leakage from Gradientscited by 1,004
  7. 2018How To Backdoor Federated Learningcited by 745
  8. 2018Speech Commands: A Dataset for Limited-Vocabulary Speech Recognitioncited by 741
  9. 2014New insights and perspectives on the natural gradient methodcited by 435
  10. 2019Robust and Communication-Efficient Federated Learning from Non-IID Datacited by 415
  11. 2019Federated Learning in Mobile Edge Networks: A Comprehensive Surveycited by 407
  12. 2019FedPAQ: A Communication-Efficient Federated Learning Method with Periodic Averaging and Quantizationcited by 349
21 more
  1. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  2. 2020Data Poisoning Attacks Against Federated Learning Systemscited by 266
  3. 2020Federated Learning with Compression: Unified Analysis and Sharp Guaranteescited by 159
  4. 2018Federated Learning for Keyword Spottingcited by 95
  5. 2020Joint Device Scheduling and Resource Allocation for Latency Constrained Wireless Federated Learningcited by 90
  6. 2019Federated Variance-Reduced Stochastic Gradient Descent with Robustness to Byzantine Attackscited by 79
  7. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  8. 2019Federated Learning for Edge Networks: Resource Optimization and Incentive Mechanismcited by 75
  9. 2020HFEL: Joint Edge Association and Resource Allocation for Cost-Efficient Hierarchical Federated Edge Learningcited by 74
  10. 2020Cost-Effective Federated Learning Designcited by 61
  11. 2017GIANT: Globally Improved Approximate Newton Method for Distributed Optimizationcited by 43
  12. 2019Accelerating DNN Training in Wireless Federated Edge Learning Systemscited by 34
  13. 2020Federated Learning for 6G Communications: Challenges, Methods, and Future Directionscited by 32
  14. 2020Coded Computing for Low-Latency Federated Learning over Wireless Edge Networkscited by 32
  15. 2016A Multi-Batch L-BFGS Method for Machine Learningcited by 30
  16. 2020When Deep Reinforcement Learning Meets Federated Learning: Intelligent Multi-Timescale Resource Management for Multi-access Edge Computing in 5G Ultra Dense Networkcited by 29
  17. 2018Towards Secure Blockchain-enabled Internet of Vehicles: Optimizing Consensus Management Using Reputation and Contract Theorycited by 14
  18. 2020FedCD: Improving Performance in non-IID Federated Learningcited by 13
  19. 2020Towards Communication-efficient and Attack-Resistant Federated Edge Learning for Industrial Internet of Thingscited by 9
  20. 2020Scalable and Communication-efficient Decentralized Federated Edge Learning with Multi-blockchain Frameworkcited by 7
  21. 2018On the Acceleration of L-BFGS with Second-Order Information and Stochastic Batchescited by 1

Built on by

Cited by year

Citing papers per year: 2022: 4; 2023: 2; 2024: 3; 2025: 1.

10 papers.

  1. 2025Federated Continual Learning: Concepts, Challenges, and Solutionscited by 13
  2. 2022SHED: A Newton-type algorithm for federated learning based on incremental Hessian eigenvector sharingcited by 11
  3. 2024Training Machine Learning models at the Edge: A Surveycited by 11
  4. 2022Robust Semi-supervised Federated Learning for Images Automatic Recognition in Internet of Dronescited by 7
  5. 2023Hierarchical Personalized Federated Learning Over Massive Mobile Edge Computing Networkscited by 7
  6. 2022Enabling All In-Edge Deep Learning: A Literature Reviewcited by 4
  7. 2023Temporal Gradient Inversion Attacks with Robust Optimizationcited by 4
  8. 2024FedVAE: Trajectory privacy preserving based on Federated Variational AutoEncodercited by 3
  9. 2024A Comprehensive Survey on Joint Resource Allocation Strategies in Federated Edge Learningcited by 1
  10. 2022Need of 6G for the Metaverse Realizationcited by 0