mlsanityData to 7 Oct 2026

The Asymmetric Harms of LLM Compression

arXiv 2608.19670 ↗First posted 20 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) compression reduces deployment costs, but standard aggregate metrics like perplexity and accuracy often mask underlying behavioral shifts. In this work, we systematically evaluate 3 LLMs across 11 compression methods to investigate the effects of compression on knowledge retention, model confidence, and social bias. We find that compression disproportionately reduces the relative retention of head knowledge compared to tail knowledge. Furthermore, compressed models often remain substantially confident in their incorrect answers on newly lost knowledge. Finally, we demonstrate that stable aggregate bias scores can conceal substantial, opposing shifts in stereotypical preferences across demographic subgroups. Together, these findings reveal asymmetric behavioral changes that aggregate performance measures fail to capture, highlighting the need for granular evaluation of compressed models before deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 36 of its 45 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

36 of 45 references.

References · 36

  1. 2016Pointer Sentinel Mixture Modelscited by 2,992
  2. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  3. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  4. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,103
  5. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  6. 2021BBQ: A Hand-Built Bias Benchmark for Question Answeringcited by 683
  7. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 657
  8. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  9. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  10. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  11. 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
  12. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
24 more
  1. 2024The Unreasonable Ineffectiveness of the Deeper Layerscited by 219
  2. 2023Head-to-Tail: How Knowledgeable are Large Language Models (LLMs)? A.K.A. Will LLMs Replace Knowledge Graphs?cited by 167
  3. 2020Characterising Bias in Compressed Modelscited by 162
  4. 2024LaCo: Large Language Model Pruning via Layer Collapsecited by 124
  5. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blockscited by 96
  6. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 94
  7. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  8. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
  9. 2022Pruning has a disparate impact on model accuracycited by 43
  10. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  11. 2024Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compressioncited by 32
  12. 2025GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidancecited by 28
  13. 2025Wanda++: Pruning Large Language Models via Regional Gradientscited by 26
  14. 2024LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Gridcited by 20
  15. 2024FrameQuant: Flexible Low-Bit Quantization for Transformerscited by 17
  16. 2025NestQuant: Nested Lattice Quantization for Matrix Products and LLMscited by 16
  17. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  18. 20252SSP: A Two-Stage Framework for Structured Pruning of LLMscited by 12
  19. 2025Why Do Some Inputs Break Low-Bit LLM Quantization?cited by 8
  20. 2026SliderQuant: Accurate Post-Training Quantization for LLMscited by 5
  21. 2026Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levelscited by 5
  22. 2024TrimLLM: Progressive Layer Dropping for Domain-Specific LLMscited by 2
  23. 2025Semantic Retention and Extreme Compression in LLMs: Can We Have Both?cited by 2
  24. 2026Weight Pruning Amplifies Bias: A Multi-Method Study of Compressed LLMs for Edge AIcited by 2

Built on by

Not cited yet.