mlsanityData to 7 Oct 2026

Rendering Wireless Environments Useful for Gradient Estimators: A Zero-Order Stochastic Federated Learning Method

arXiv 2401.17460 ↗First posted 30 Jan 2024cs.LG cs.DC cs.MAcited by 4

Not on the map. Why?

Abstract (arXiv)

Cross-device federated learning (FL) is a growing machine learning setting whereby multiple edge devices collaborate to train a model without disclosing their raw data. With the great number of mobile devices participating in more FL applications via the wireless environment, the practical implementation of these applications will be hindered due to the limited uplink capacity of devices, causing critical bottlenecks. In this work, we propose a novel doubly communication-efficient zero-order (ZO) method with a one-point gradient estimator that replaces communicating long vectors with scalar values and that harnesses the nature of the wireless communication channel, overcoming the need to know the channel state coefficient. It is the first method that includes the wireless channel in the learning algorithm itself instead of wasting resources to analyze it and remove its impact. We then offer a thorough analysis of the proposed zero-order federated learning (ZOFL) framework and prove that our method converges textitalmost surely, which is a novel result in nonconvex ZO optimization. We further prove a convergence rate of O(frac1√[3]K) in the nonconvex setting. We finally demonstrate the potential of our algorithm with experimental results.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 29 of its 42 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

29 of 42 references.

References · 29

  1. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
  2. 2019Federated Learning: Challenges, Methods, and Future Directionscited by 1,531
  3. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  4. 2019Towards Federated Learning at Scale: System Designcited by 826
  5. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  6. 2019Tighter Theory for Local SGD on Identical and Heterogeneous Datacited by 266
  7. 2018Federated Learning via Over-the-Air Computationcited by 215
  8. 2013Optimal rates for zero-order convex optimization: the power of two function evaluationscited by 190
  9. 2019Federated Learning over Wireless Fading Channelscited by 178
  10. 2020FedPD: A Federated Learning Framework with Optimal Rates and Adaptivity to Non-IID Datacited by 143
  11. 2019Distributed Learning with Compressed Gradient Differencescited by 110
  12. 2018LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learningcited by 95
17 more
  1. 2020Federated Bayesian Optimization via Thompson Samplingcited by 77
  2. 2020FedDANE: A Federated Newton-Type Methodcited by 76
  3. 2019ZO-AdaMM: Zeroth-Order Adaptive Momentum Method for Black-Box Optimizationcited by 75
  4. 2020Over-the-Air Federated Learning from Heterogeneous Datacited by 72
  5. 2020Convergence of Update Aware Device Scheduling for Federated Learning at the Wireless Edgecited by 71
  6. 2019On Analog Gradient Descent Learning over Multiple Access Fading Channelscited by 61
  7. 2022Communication-Efficient Stochastic Zeroth-Order Optimization for Federated Learningcited by 59
  8. 2018Derivative-Free Methods for Policy Optimization: Guarantees for Linear Quadratic Systemscited by 56
  9. 2018Distributed learning with compressed gradientscited by 53
  10. 2019Model Agnostic Contrastive Explanations for Structured Datacited by 41
  11. 2021Dynamic Scheduling for Over-the-Air Federated Edge Learning with Energy Constraintscited by 34
  12. 2019Contrasting Exploration in Parameter and Action Space: A Zeroth-Order Optimization Perspectivecited by 26
  13. 2022FedNew: A Communication-Efficient and Privacy-Preserving Newton-Type Method for Federated Learningcited by 23
  14. 2019Q-GADMM: Quantized Group ADMM for Communication Efficient Decentralized Machine Learningcited by 17
  15. 2019Making Cell-Free Massive MIMO Competitive With MMSE Processing and Centralized Implementationcited by 13
  16. 2022Distributed Learning With Sparsified Gradient Differencescited by 9
  17. 2022Zero-Order One-Point Estimate with Distributed Stochastic Gradient-Tracking Techniquecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Communication and Energy Efficient Federated Learning using Zero-Order Optimization Techniquecited by 1
  2. 2024Fed-ZOE: Communication-Efficient Over-the-Air Federated Learning via Zeroth-Order Estimationcited by 1
  3. 2025Communication-Efficient Zero-Order and First-Order Federated Learning Methods over Wireless Networkscited by 1
  4. 2026AirGC-CD: Gaussian-Circulant Precoding for Exactly Debiasable PAPR Reduction in Over-the-Air Federated Learningcited by 0