mlsanityData to 14 Oct 2026

Personalized Federated Learning by Structured and Unstructured Pruning under Data Heterogeneity

arXiv 2105.00562 ↗First posted 2 May 2021cs.LG cs.DCcited by 27

Not on the map. Why?

Abstract (arXiv)

The traditional approach in FL tries to learn a single global model collaboratively with the help of many clients under the orchestration of a central server. However, learning a single global model might not work well for all clients participating in the FL under data heterogeneity. Therefore, the personalization of the global model becomes crucial in handling the challenges that arise with statistical heterogeneity and the non-IID distribution of data. Unlike prior works, in this work we propose a new approach for obtaining a personalized model from a client-level objective. This further motivates all clients to participate in federation even under statistical heterogeneity in order to improve their performance, instead of merely being a source of data and model training for the central server. To realize this personalization, we leverage finding a small subnetwork for each client by applying hybrid pruning (combination of structured and unstructured pruning), and unstructured pruning. Through a range of experiments on different benchmarks, we observed that the clients with similar data (labels) share similar personal parameters. By finding a subnetwork for each client ...

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 2 May 2021.

Builds on

26 of 37 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  3. 2019Advances and Open Problems in Federated Learningcited by 3,046
  4. 2018Federated Optimization in Heterogeneous Networkscited by 2,987
  5. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  6. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  7. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,475
  8. 2016Pruning Filters for Efficient ConvNetscited by 1,143
  9. 2018Federated Learning with Non-IID Datacited by 1,108
  10. 2016Deep Networks with Stochastic Depthcited by 979
  11. 2019A Closer Look at Few-shot Classificationcited by 684
  12. 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
14 more
  1. 2017Federated Multi-Task Learningcited by 672
  2. 2018Federated Learning for Mobile Keyboard Predictioncited by 572
  3. 2016Learning Structured Sparsity in Deep Neural Networkscited by 541
  4. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 478
  5. 2020Think Locally, Act Globally: Federated Learning with Local and Global Representationscited by 325
  6. 2020Three Approaches for Personalization with Applications to Federated Learningcited by 325
  7. 2019Improving Federated Learning Personalization via Model Agnostic Meta Learningcited by 269
  8. 2020Federated Learning of a Mixture of Global and Local Modelscited by 210
  9. 2018Federated Meta-Learning with Fast Convergence and Efficient Communicationcited by 155
  10. 2019Federated Evaluation of On-device Personalizationcited by 147
  11. 2020Salvaging Federated Learning by Local Adaptationcited by 142
  12. 2018Geometric Lower Bounds for Distributed Parameter Estimation under Communication Constraintscited by 33
  13. 2019An Investigation Into On-device Personalization of End-to-end Automatic Speech Recognition Modelscited by 25
  14. 2019Robust and Communication-Efficient Collaborative Learningcited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 7; 2023: 8; 2024: 7; 2025: 3; 2026: 2.

27 papers.

Sort:

By area:

  1. 2023Towards Building the Federated GPT: Federated Instruction Tuningcited by 185
  2. 2022DisPFL: Towards Communication-Efficient Personalized Federated Learning via Decentralized Sparse Trainingcited by 74
  3. 2022Efficient Distribution Similarity Identification in Clustered Federated Learning via Principal Angles Between Client Data Subspacescited by 36
  4. 2022Achieving Personalized Federated Learning with Sparse Local Modelscited by 31
  5. 2022Rethinking Data Heterogeneity in Federated Learning: Introducing a New Notion and Standard Benchmarkscited by 24
  6. 2022FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distributioncited by 22
  7. 2023Unlocking the Potential of Federated Learning: The Symphony of Dataset Distillation via Deep Generative Latentscited by 16
  8. 2023AEDFL: Efficient Asynchronous Decentralized Federated Learning with Heterogeneous Devicescited by 15
  9. 2023Complement Sparsification: Low-Overhead Model Pruning for Federated Learningcited by 12
  10. 2022When Do Curricula Work in Federated Learning?cited by 9
  11. 2025FedConv: A Learning-on-Model Paradigm for Heterogeneous Federated Clientscited by 7
  12. 2023FedDebug: Systematic Debugging for Federated Learning Applicationscited by 6
15 more
  1. 2023A Practical Recipe for Federated Learning Under Statistical Heterogeneity Experimental Designcited by 5
  2. 2024Federated Learning Client Pruning for Noisy Labelscited by 4
  3. 2023CEFHRI: A Communication Efficient Federated Learning Framework for Recognizing Industrial Human-Robot Interactioncited by 3
  4. 2022Self-Programming Artificial Intelligence Using Code-Generating Language Modelscited by 2
  5. 2023Cross-Silo Federated Learning Across Divergent Domains with Iterative Parameter Alignmentcited by 1
  6. 2024Rapid Deployment of DNNs for Edge Computing via Structured Pruning at Initializationcited by 1
  7. 2024SSFL: Discovering Sparse Unified Subnetworks at Initialization for Efficient Federated Learningcited by 1
  8. 2024Federated Learning with Flexible Architecturescited by 1
  9. 2024Automatic Structured Pruning for Efficient Architecture in Federated Learningcited by 1
  10. 2025FedBKD: Distilled Federated Learning to Embrace Gerneralization and Personalization on Non-IID Datacited by 1
  11. 2024Federated Learning based on Pruning and Recoverycited by 0
  12. 2024subMFL: Compatiple subModel Generation for Federated Learning in Device Heterogenous Environmentcited by 0
  13. 2025Evaluating Small Language Models for Agentic On-Farm Decision Support Systemscited by 0
  14. 2026FedImp: Enhancing Federated Learning Convergence with Impurity-Based Weightingcited by 0
  15. 2026FedASAP: Activation Statistics-driven Structured Adaptive Pruning for Efficient Personalized Federated Learning for Lesion Segmentation on brain MRIcited by 0