Automatic Structured Pruning for Efficient Architecture in Federated Learning
Not on the map. Why?
Abstract (arXiv)
In Federated Learning (FL), training is conducted on client devices, typically with limited computational resources and storage capacity. To address these constraints, we propose an automatic pruning scheme tailored for FL systems. Our solution improves computation efficiency on client devices, while minimizing communication costs. One of the challenges of tuning pruning hyper-parameters in FL systems is the restricted access to local data. Thus, we introduce an automatic pruning paradigm that dynamically determines pruning boundaries. Additionally, we utilized a structured pruning algorithm optimized for mobile devices that lack hardware support for sparse computations. Experimental results demonstrate the effectiveness of our approach, achieving accuracy comparable to existing methods. Our method notably reduces the number of parameters by 89% and FLOPS by 90%, with minimal impact on the accuracy of the FEMNIST and CelebFaces datasets. Furthermore, our pruning method decreases communication overhead by up to 5x and halves inference time when deployed on Android devices.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 24 references that are in the map. First posted on arXiv 4 Nov 2024.
Builds on
16 of 24 references.
References · 16
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
- 2014Going Deeper with Convolutionscited by 4,241
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2016Pruning Filters for Efficient ConvNetscited by 1,143
- 2018LEAF: A Benchmark for Federated Settingscited by 792
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
- 2018Rethinking the Value of Network Pruningcited by 599
- 2021FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learningcited by 141
- 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
- 2021Federated Dynamic Sparse Training: Computing Less, Communicating Less, Yet Learning Bettercited by 55
- 2022ZeroFL: Efficient On-Device Training for Federated Learning with Local Sparsitycited by 45
- 2022Sparse Random Networks for Communication-Efficient Federated Learningcited by 32
4 more
- 2022FedDUAP: Federated Learning with Dynamic Update and Adaptive Pruning Using Shared Data on the Servercited by 29
- 2021Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneitycited by 27
- 2022Distributed Pruning Towards Tiny Neural Networks in Federated Learningcited by 11
- 2024FedLPS: Heterogeneous Federated Learning for Multiple Tasks with Local Parameter Sharingcited by 8
Built on by
Citing papers per year: 2025: 1.
1 paper.