mlsanityData to 7 Oct 2026

On the Impact of Calibration Data in Post-training Quantization and Pruning

arXiv 2311.09755 ↗First posted 16 Nov 2023cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Quantization and pruning form the foundation of compression for neural networks, enabling efficient inference for large language models (LLMs). Recently, various quantization and pruning techniques have demonstrated remarkable performance in a post-training setting. They rely upon calibration data, a small set of unlabeled examples that are used to generate layer activations. However, no prior work has systematically investigated how the calibration data impacts the effectiveness of model compression methods. In this paper, we present the first extensive empirical study on the effect of calibration data upon LLM performance. We trial a variety of quantization and pruning methods, datasets, tasks, and models. Surprisingly, we find substantial variations in downstream task performance, contrasting existing work that suggests a greater level of robustness to the calibration data. Finally, we make a series of recommendations for the effective use of calibration data in LLM quantization and pruning.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Builds on 33 of its 52 references that are in the map. First posted on arXiv 16 Nov 2023.

Builds on

33 of 52 references.

References · 33

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2016Pointer Sentinel Mixture Modelscited by 2,992
  8. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  9. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  10. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  11. 2017Get To The Point: Summarization with Pointer-Generator Networkscited by 1,281
  12. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
21 more
  1. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  2. 2015Teaching Machines to Read and Comprehendcited by 1,039
  3. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  4. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  5. 2019Green AIcited by 620
  6. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  7. 2016Learning Structured Sparsity in Deep Neural Networkscited by 538
  8. 2021Datasets: A Community Library for Natural Language Processingcited by 526
  9. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  10. 2016A Corpus and Evaluation Framework for Deeper Understanding of Commonsense Storiescited by 413
  11. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  12. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  13. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 247
  14. 2021Accelerating Sparse Deep Neural Networkscited by 213
  15. 2021Efficient Deep Learning: A Survey on Making Deep Learning Models Smaller, Faster, and Bettercited by 153
  16. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  17. 2023Intriguing Properties of Quantization at Scalecited by 48
  18. 2023Pruning vs Quantization: Which is Better?cited by 45
  19. 2019Careful Selection of Knowledge to solve Open Book Question Answeringcited by 35
  20. 2023Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantizationcited by 20
  21. 2023Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarizationcited by 16

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 8; 2025: 21; 2026: 21.

51 papers.

Sort:

By area:

  1. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 68
  2. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  3. 2024Reassessing Layer Pruning in LLMs: New Insights and Methodscited by 23
  4. 2023ZeroQuant(4+2): Redefining LLMs Quantization with a New FP6-Centric Strategy for Diverse Generative Taskscited by 14
  5. 2024Beware of Calibration Data for Pruning Large Language Modelscited by 14
  6. 2024Outliers and Calibration Sets have Diminishing Effect on Quantization of Modern LLMscited by 11
  7. 2024Evaluating the Generalization Ability of Quantized LLMs: Benchmark, Analysis, and Toolboxcited by 11
  8. 2024Self-calibration for Language Model Quantization and Pruningcited by 10
  9. 2025Does quantization affect models' performance on long-context tasks?cited by 10
  10. 2025Preserving LLM Capabilities through Calibration Data Curation: From Analysis to Optimizationcited by 9
  11. 2024On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruningcited by 6
  12. 2025Task-Circuit Quantization: Leveraging Knowledge Localization and Interpretability for Compressioncited by 6
38 more
  1. 2025NIRVANA: Structured Pruning Reimagined for Large Language Model Compressioncited by 6
  2. 2026Beyond Outliers: A Data-Free Layer-wise Mixed-Precision Quantization Approach Driven by Numerical and Structural Dual-Sensitivitycited by 5
  3. 2025Compressing Language Models for Specialized Domainscited by 4
  4. 2025Mitigating Catastrophic Forgetting in Target Language Adaptation of LLMs via Source-Shielded Updatescited by 4
  5. 2025Revisiting Large Language Model Pruning using Neuron Semantic Attributioncited by 3
  6. 2025μ-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Expertscited by 3
  7. 2026OneComp: One-Line Revolution for Generative AI Model Compressioncited by 3
  8. 2025M-Wanda: Improving One-Shot Pruning for Multilingual LLMscited by 2
  9. 2025Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinionscited by 2
  10. 2025Task-Stratified Knowledge Scaling Laws for Post-Training Quantized Large Language Modelscited by 2
  11. 2026Calibrating Beyond English: Language Diversity for Better Quantized Multilingual LLMcited by 2
  12. 2026Rethinking Practical and Efficient Quantization Calibration for Vision-Language Modelscited by 2
  13. 2026Understanding Calibration and Truncation Error Propagation in Training-Free Low-Rank Compression for LLMscited by 2
  14. 2024Enhancing One-shot Pruned Pre-trained Language Models through Sparse-Dense-Sparse Mechanismcited by 1
  15. 2025Code-Driven Inductive Synthesis: Enhancing Reasoning Abilities of Large Language Models with Sequencescited by 1
  16. 2025CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Roundingcited by 1
  17. 2025EfficientXpert: Efficient Domain Adaptation for Large Language Models via Propagation-Aware Pruningcited by 1
  18. 2026GPrune-LLM: Generalization-Aware Structured Pruning for Large Language Modelscited by 1
  19. 2026Frequency Matters: Fast Model-Agnostic Data Curation for Pruning and Quantizationcited by 1
  20. 2026LAQuant: A Simple Overhead-free Large Reasoning Model Quantization by Layer-wise Lookahead Losscited by 1
  21. 2026The Structure of Quantization Damage in LLMs: Why the Next Bit Should Be Spent Globallycited by 1
  22. 2025Deriving Coding-Specific Sub-Models from LLMs using Resource-Efficient Pruningcited by 0
  23. 2025Q&C: When Quantization Meets Cache in Efficient Image Generationcited by 0
  24. 2025As easy as PIE: understanding when pruning causes language models to disagreecited by 0
  25. 2025Post-Training Quantization of Generative and Discriminative LSTM Text Classifiers: A Study of Calibration, Class Balance, and Robustnesscited by 0
  26. 2025Fair-GPTQ: Bias-Aware Quantization for Large Language Modelscited by 0
  27. 2025Leveraging KV Similarity for Online Structured Pruning in LLMscited by 0
  28. 2026Meeting SLOs, Slashing Hours: Automated Enterprise LLM Optimization with OptiKITcited by 0
  29. 2026UniComp: A Unified Evaluation of Large Language Model Compression via Pruning, Quantization, and Distillationcited by 0
  30. 2026Robust Ultra Low-Bit Post-Training Quantization via Stable Diagonal Curvature Estimatecited by 0
  31. 2026Coverage-Based Calibration for Post-Training Quantization via Weighted Set Cover over Outlier Channelscited by 0
  32. 2026Averaged Evaluation Masks Capability Trade-Offs: Multi-Source Calibration for High-Sparsity LLM Pruningcited by 0
  33. 2026EinSort: Sorting is All We Need for Tensorizing LLMcited by 0
  34. 2026Calibration Bets on the Past: Post-Training Quantization for Financial Time-Series Forecastingcited by 0
  35. 2026Target-Aware Calibration Data Selection for Preserving Uncertainty in Quantized Language Modelscited by 0
  36. 2026Hardware-Accelerated Instance Segmentation for Resource-Constrained Space Robotics with Criticality Analysiscited by 0
  37. 2026When Quantization Preserves Accuracy but Not Evidence: Explanation-Aware Post-Training Quantization for Medical LLMscited by 0
  38. 2026Damage Predicts Recovery: When Calibration Data Matters in Compressing Financial LLMscited by 0

50 most cited of 51 All 51 (JSON)