mlsanityData to 7 Oct 2026

Gradient-Free Structured Pruning with Unlabeled Data

arXiv 2303.04185 ↗First posted 7 Mar 2023cs.LG cs.AI cs.CLcited by 26

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have achieved great success in solving difficult tasks across many domains, but such success comes with a high computation cost, and inference latency. As developers and third parties customize these models, the need to provide efficient inference has increased. Many efforts have attempted to reduce inference cost through model compression techniques such as pruning and distillation. However, these techniques either require labeled data, or are time-consuming as they require the compressed model to be retrained to regain accuracy. In this paper, we propose a gradient-free structured pruning framework that uses only unlabeled data. An evaluation on the GLUE and SQuAD benchmarks using BERTBASE and DistilBERT illustrates the effectiveness of the proposed approach. By only using the weights of the pre-trained model and unlabeled data, in a matter of a few minutes on a single GPU, up to 40% of the original FLOP count can be reduced with less than a 4% accuracy loss across all tasks considered.

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 72 of its 100 references that are in the map. First posted on arXiv 7 Mar 2023.

Builds on

72 of 100 references.

References · 72

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,665
  9. 2020Reformer: The Efficient Transformercited by 1,658
  10. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  11. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  12. 2019TinyBERT: Distilling BERT for Natural Language Understandingcited by 1,153
38 more
  1. 2016Pruning Filters for Efficient ConvNetscited by 1,129
  2. 2017SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluationcited by 998
  3. 2018Neural Network Acceptability Judgmentscited by 952
  4. 2019Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Prunedcited by 844
  5. 2019Are Sixteen Heads Really Better than One?cited by 803
  6. 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
  7. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 476
  8. 2019The State of Sparsity in Deep Neural Networkscited by 471
  9. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  10. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 398
  11. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  12. 2020DeeBERT: Dynamic Early Exiting for Accelerating BERT Inferencecited by 322
  13. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  14. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  15. 2020The Lottery Ticket Hypothesis for Pre-trained BERT Networkscited by 275
  16. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 267
  17. 2021I-BERT: Integer-only BERT Quantizationcited by 250
  18. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 246
  19. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  20. 2019Structured Pruning of Large Language Modelscited by 201
  21. 2021Chasing Sparsity in Vision Transformers: An End-to-End Explorationcited by 181
  22. 2019The Evolved Transformercited by 180
  23. 2021Block Pruning For Faster Transformerscited by 176
  24. 2022Structured Pruning Learns Compact and Accurate Modelscited by 167
  25. 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
  26. 2022A Fast Post-Training Pruning Framework for Transformerscited by 161
  27. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  28. 2020Lite Transformer with Long-Short Range Attentioncited by 127
  29. 2020On the Effect of Dropping Layers of Pre-trained Transformer Modelscited by 118
  30. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  31. 2020When BERT Plays the Lottery, All Tickets Are Winningcited by 114
  32. 2020The Effect of Natural Distribution Shift on Question Answering Modelscited by 104
  33. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 103
  34. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  35. 2019Provable Filter Pruning for Efficient Neural Networkscited by 98
  36. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 95
  37. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 89
  38. 2020EarlyBERT: Efficient BERT Training via Early-bird Lottery Ticketscited by 79

50 most cited of 72 All 72 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 4; 2024: 10; 2025: 9; 2026: 3.

26 papers.

Sort:

By area:

  1. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  2. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  3. 2024Shortened LLaMA: Depth Pruning for Large Language Models with Comparison of Retraining Methodscited by 80
  4. 2024Everybody Prune Now: Structured Pruning of LLMs with only Forward Passescited by 42
  5. 2024A Survey of Lottery Ticket Hypothesiscited by 23
  6. 2024Pruning Large Language Models with Semi-Structural Adaptive Sparse Trainingcited by 20
  7. 2024A Comprehensive Survey of Compression Algorithms for Language Modelscited by 15
  8. 2023Accurate Retraining-free Pruning for Pretrained Encoder-based Language Modelscited by 11
  9. 2025Sliding-Window Merging for Compacting Patch-Redundant Layers in LLMscited by 8
  10. 2025Short-LVLM: Compressing and Accelerating Large Vision-Language Models by Pruning Redundant Layerscited by 7
  11. 2023rm SP3: Enhancing Structured Pruning via PCA Projectioncited by 5
  12. 2024MINI-LLM: Memory-Efficient Structured Pruning for Large Language Modelscited by 5
14 more
  1. 2025The Right to be Forgotten in Pruning: Unveil Machine Unlearning on Sparse Modelscited by 5
  2. 2024Reconstruct the Pruned Model without Any Retrainingcited by 4
  3. 2026Prune-then-Quantize or Quantize-then-Prune? Understanding the Impact of Compression Order in Joint Model Compressioncited by 2
  4. 2024Large Language Model Pruningcited by 1
  5. 2025Harnessing Input-Adaptive Inference for Efficient VLNcited by 1
  6. 2025SlimDiff: Training-Free, Activation-Guided Hands-free Slimming of Diffusion Modelscited by 1
  7. 2025CAST: Continuous and Differentiable Semi-Structured Sparsity-Aware Training for Large Language Modelscited by 1
  8. 2025Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptationcited by 1
  9. 2025Frustratingly Easy Task-aware Pruning for Large Language Modelscited by 1
  10. 2024Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Modelscited by 0
  11. 2024Inducing Semi-Structured Sparsity by Masking for Efficient Model Inference in Convolutional Networkscited by 0
  12. 2025Efficient Large Language Models with Zero-Shot Adjustable Accelerationcited by 0
  13. 2026CORP: Closed-Form One-shot Representation-Preserving Structured Pruning for Transformerscited by 0
  14. 2026Compositionality Emerges in a Narrow Depth-Connectivity Regime: Architecture Constraints and Solution Manifoldscited by 0