mlsanityData to 14 Oct 2026

Exploring Federated Pruning for Large Language Models

arXiv 2505.13547 ↗First posted 19 May 2025cs.LG cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

LLM pruning has emerged as a promising technology for compressing LLMs, enabling their deployment on resource-limited devices. However, current methodologies typically require access to public calibration samples, which can be challenging to obtain in privacy-sensitive domains. To address this issue, we introduce FedPrLLM, a comprehensive federated pruning framework designed for the privacy-preserving compression of LLMs. In FedPrLLM, each client only needs to calculate a pruning mask matrix based on its local calibration data and share it with the server to prune the global model. This approach allows for collaborative pruning of the global model with the knowledge of each client while maintaining local data privacy. Additionally, we conduct extensive experiments to explore various possibilities within the FedPrLLM framework, including different comparison groups, pruning strategies, and the decision to scale weights. Our extensive evaluation reveals that one-shot pruning with layer comparison and no weight scaling is the optimal choice within the FedPrLLM framework. We hope our work will help guide future efforts in pruning LLMs in privacy-sensitive fields. Our code is available at https://github.com/Pengxin-Guo/FedPrLLM.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 33 references that are in the map. First posted on arXiv 19 May 2025.

Builds on

29 of 33 references.

References · 29

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,669
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  10. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  11. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  12. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
17 more
  1. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  2. 2024GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projectioncited by 463
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  4. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 415
  5. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  6. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 284
  7. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 226
  8. 2023Outlier Weighed Layerwise Sparsity (OWL): A Missing Secret Sauce for Pruning LLMs to High Sparsitycited by 165
  9. 2023LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximationcited by 104
  10. 2024What Matters in Transformers? Not All Attention is Neededcited by 84
  11. 2024Selective Aggregation for Low-Rank Adaptation in Federated Learningcited by 80
  12. 2024BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocationcited by 45
  13. 2023Matrix Compression via Randomized Low Rank and Low Precision Factorizationcited by 25
  14. 2024Tackling Data Heterogeneity in Federated Learning via Loss Decompositioncited by 9
  15. 2025Exploring the Vulnerabilities of Federated Learning: A Deep Dive into Gradient Inversion Attackscited by 9
  16. 2024A New Federated Learning Framework Against Gradient Inversion Attackscited by 6
  17. 2024FedSpaLLM: Federated Pruning of Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Beyond Denial-of-Service: The Puppeteer's Attack for Fine-Grained Control in Ranking-Based Federated Learningcited by 1
  2. 2026Fed-GRPO: Reward-Signal-Driven Federated Group Relative Policy Optimizationcited by 0