mlsanityData to 7 Oct 2026

F-WANDA: Fisher-Reweighted Post-Training Pruning for Sustainable Deployment of Large Language Models

arXiv 2608.00481 ↗First posted 1 Aug 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

One-shot post-training pruning is the most energy-frugal compression strategy for largelanguage models (LLMs), yet existing approaches trade either quality (WANDA) or compute cost (SPARSEGPT). We introduce F-WANDA, a drop-in modification of WANDA that reallocates the per-row keep budget across output neurons in proportion to the empirical Fisher information of the pre-activation. The Fisher signal is collected in a single additional backward pass over the same calibration corpus WANDA already uses; no weights are updated. On LLAMA-2-7B at 50 % unstructured sparsity, F-WANDA attains WikiText-2 perplexity of 6.85, matches WANDA fluency, and improves 5-shot MMLU by +1.6 pp over WANDA and +1.1 pp over SPARSEGPT, while incurring only one-third of SPARSEGPT pruning wall-clock and energy. The headline trade-off is achieved without extra calibration data or fine-tuning, placing F-WANDA on the Pareto frontier of quality versus pruning cost for sustainable LLM compression.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 24 references that are in the map. First posted on arXiv 1 Aug 2026.

Builds on

20 of 24 references.

References · 20

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  6. 2016Pointer Sentinel Mixture Modelscited by 2,992
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  9. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  10. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  11. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  12. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
8 more
  1. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  2. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  3. 2019Green AIcited by 620
  4. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  5. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  6. 2021Accelerating Sparse Deep Neural Networkscited by 213
  7. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  8. 2018Faster gaze prediction with dense networks and Fisher pruningcited by 92

Built on by

Not cited yet.