# Accelerating Wireless Federated Learning via Nesterov's Momentum and Distributed Principle Component Analysis

arXiv: https://arxiv.org/abs/2303.17885 · HTML: https://mlsanity.com/paper/2303.17885/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 54 references that are in the map.
- First posted on arXiv 31 Mar 2023.

## Builds on (most cited first, 30 of 32)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [A Joint Learning and Communications Framework for Federated Learning over Wireless Networks](https://mlsanity.com/paper/1909.07972/) (2019)
- [Federated Learning via Over-the-Air Computation](https://mlsanity.com/paper/1812.11750/) (2018)
- [Broadband Analog Aggregation for Low-Latency Federated Edge Learning (Extended Version)](https://mlsanity.com/paper/1812.11494/) (2018)
- [Federated Learning over Wireless Fading Channels](https://mlsanity.com/paper/1907.09769/) (2019)
- [Energy Efficient Federated Learning Over Wireless Communication Networks](https://mlsanity.com/paper/1911.02417/) (2019)
- [On the Convergence of A Class of Adam-Type Algorithms for Non-Convex Optimization](https://mlsanity.com/paper/1808.02941/) (2018)
- [Convergence of Edge Computing and Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/1907.08349/) (2019)
- [One-Bit Over-the-Air Aggregation for Communication-Efficient Federated Edge Learning: Design and Convergence Analysis](https://mlsanity.com/paper/2001.05713/) (2020)
- [Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective](https://mlsanity.com/paper/2004.04314/) (2020)
- [LAG: Lazily Aggregated Gradient for Communication-Efficient Distributed Learning](https://mlsanity.com/paper/1805.09965/) (2018)
- [Towards an Intelligent Edge: Wireless Communication Meets Machine Learning](https://mlsanity.com/paper/1809.00343/) (2018)
- [Turning Big data into tiny data: Constant-size coresets for k-means, PCA and projective clustering](https://mlsanity.com/paper/1807.04518/) (2018)
- [Lipschitz constant estimation of Neural Networks via sparse polynomial optimization](https://mlsanity.com/paper/2004.08688/) (2020)
- [Over-the-Air Federated Learning from Heterogeneous Data](https://mlsanity.com/paper/2009.12787/) (2020)
- [Privacy For Free: Wireless Federated Learning Via Uncoded Transmission With Adaptive Power Control](https://mlsanity.com/paper/2006.05459/) (2020)
- [On Analog Gradient Descent Learning over Multiple Access Fading Channels](https://mlsanity.com/paper/1908.07463/) (2019)
- [Wireless Networks Design in the Era of Deep Learning: Model-Based, AI-Based, or Both?](https://mlsanity.com/paper/1902.02647/) (2019)
- [Accelerating DNN Training in Wireless Federated Edge Learning Systems](https://mlsanity.com/paper/1905.09712/) (2019)
- [High-Dimensional Stochastic Gradient Quantization for Communication-Efficient Edge Learning](https://mlsanity.com/paper/1910.03865/) (2019)
- [Federated Edge Learning with Misaligned Over-The-Air Computation](https://mlsanity.com/paper/2102.13604/) (2021)
- [Revisiting Normalized Gradient Descent: Fast Evasion of Saddle Points](https://mlsanity.com/paper/1711.05224/) (2017)
- [Federated Learning in Unreliable and Resource-Constrained Cellular Wireless Networks](https://mlsanity.com/paper/2012.05137/) (2020)
- [On the Influence of Momentum Acceleration on Online Learning](https://mlsanity.com/paper/1603.04136/) (2016)
- [Improved Distributed Principal Component Analysis](https://mlsanity.com/paper/1408.5823/) (2014)
- [Turning Channel Noise into an Accelerator for Over-the-Air Principal Component Analysis](https://mlsanity.com/paper/2104.10095/) (2021)
- [Accelerated Gradient Descent Learning over Multiple Access Fading Channels](https://mlsanity.com/paper/2107.12452/) (2021)
- [Bridging Differential Privacy and Byzantine-Robustness via Model Aggregation](https://mlsanity.com/paper/2205.00107/) (2022)
- [Reduced-Dimension Design of MIMO Over-the-Air Computing for Data Aggregation in Clustered IoT Networks](https://mlsanity.com/paper/1812.02373/) (2018)
- [Secure Distributed On-Device Learning Networks With Byzantine Adversaries](https://mlsanity.com/paper/1906.00887/) (2019)

## Built on by (most cited first, 1 of 1)

- [Communication-Efficient Large-Scale Distributed Deep Learning: A Comprehensive Survey](https://mlsanity.com/paper/2404.06114/) (2024)

## Abstract (arXiv)

A wireless federated learning system is investigated by allowing a server and workers to exchange uncoded information via orthogonal wireless channels. Since the workers frequently upload local gradients to the server via bandwidth-limited channels, the uplink transmission from the workers to the server becomes a communication bottleneck. Therefore, a one-shot distributed principle component analysis (PCA) is leveraged to reduce the dimension of uploaded gradients such that the communication bottleneck is relieved. A PCA-based wireless federated learning (PCA-WFL) algorithm and its accelerated version (i.e., PCA-AWFL) are proposed based on the low-dimensional gradients and the Nesterov's momentum. For the non-convex loss functions, a finite-time analysis is performed to quantify the impacts of system hyper-parameters on the convergence of the PCA-WFL and PCA-AWFL algorithms. The PCA-AWFL algorithm is theoretically certified to converge faster than the PCA-WFL algorithm. Besides, the convergence rates of PCA-WFL and PCA-AWFL algorithms quantitatively reveal the linear speedup with respect to the number of workers over the vanilla gradient descent algorithm. Numerical results are used to demonstrate the improved convergence rates of the proposed PCA-WFL and PCA-AWFL algorithms over the benchmarks.