mlsanityData to 7 Oct 2026

Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning

arXiv 2208.11580 ↗First posted 24 Aug 2022cs.LGcited by 317

Not on the map. Why?

Abstract (arXiv)

We consider the problem of model compression for deep neural networks (DNNs) in the challenging one-shot/post-training setting, in which we are given an accurate trained model, and must compress it without any retraining, based only on a small amount of calibration input data. This problem has become popular in view of the emerging software and hardware support for executing models compressed via pruning and/or quantization with speedup, and well-performing solutions have been proposed independently for both compression approaches. In this paper, we introduce a new compression framework which covers both weight pruning and quantization in a unified setting, is time- and space-efficient, and considerably improves upon the practical performance of existing post-training methods. At the technical level, our approach is based on an exact and efficient realization of the classical Optimal Brain Surgeon (OBS) framework of [LeCun, Denker, and Solla, 1990] extended to also cover weight quantization at the scale of modern DNNs. From the practical perspective, our experimental results show that it can improve significantly upon the compression-accuracy trade-offs of existing post-training methods, and that it can enable the accurate compound application of both pruning and quantization in a post-training setting.

Key facts. Cited by 317 papers in the map (data through 7 Oct 2026). Builds on 31 of its 48 references that are in the map. First posted on arXiv 24 Aug 2022.

Builds on

31 of 48 references.

References · 31

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  6. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  7. 2015Optimizing Neural Networks with Kronecker-factored Approximate Curvaturecited by 776
  8. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  10. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 596
  11. 2021Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networkscited by 521
  12. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
19 more
  1. 2021A White Paper on Neural Network Quantizationcited by 453
  2. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  3. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 375
  4. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  5. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 259
  6. 2017Learning to Prune Deep Neural Networks via Layer-wise Optimal Brain Surgeoncited by 219
  7. 2021Accelerating Sparse Deep Neural Networkscited by 213
  8. 2019MLPerf Inference Benchmarkcited by 195
  9. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  10. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 177
  11. 2016A Kronecker-factored approximate Fisher matrix for convolution layerscited by 138
  12. 2021Group Fisher Pruning for Practical Network Compressioncited by 130
  13. 2022The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Modelscited by 115
  14. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  15. 2019EigenDamage: Structured Pruning in the Kronecker-Factored Eigenbasiscited by 98
  16. 2019Loss Aware Post-training Quantizationcited by 94
  17. 2019Fast Sparse ConvNetscited by 86
  18. 2021M-FAC: Efficient Matrix-Free Approximations of Second-Order Informationcited by 54
  19. 2022SPDY: Accurate Pruning with Speedup Guaranteescited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 47; 2024: 99; 2025: 96; 2026: 72.

317 papers.

Sort:

By area:

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  3. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  4. 2023A Comprehensive Overview of Large Language Modelscited by 639
  5. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  6. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
  7. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  8. 2024A Survey on Knowledge Distillation of Large Language Modelscited by 276
  9. 2023A Survey on Model Compression for Large Language Modelscited by 263
  10. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
  11. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  12. 2023Efficient Large Language Models: A Surveycited by 183
38 more
  1. 2024BiLLM: Pushing the Limit of Post-Training Quantization for LLMscited by 137
  2. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  3. 2024SLEB: Streamlining LLMs through Redundancy Verification and Elimination of Transformer Blockscited by 96
  4. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  5. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 91
  6. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  7. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 78
  8. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 78
  9. 2023Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformerscited by 67
  10. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  11. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  12. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 61
  13. 2024Model Tailor: Mitigating Catastrophic Forgetting in Multi-modal Large Language Modelscited by 54
  14. 2024Decoding Compressed Trust: Scrutinizing the Trustworthiness of Efficient LLMs Under Compressioncited by 54
  15. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 54
  16. 2023On the Impact of Calibration Data in Post-training Quantization and Pruningcited by 51
  17. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 49
  18. 2024RoSA: Accurate Parameter-Efficient Fine-Tuning via Robust Adaptationcited by 48
  19. 2023ZipLM: Inference-Aware Structured Pruning of Language Modelscited by 46
  20. 2023Pruning vs Quantization: Which is Better?cited by 45
  21. 2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 45
  22. 2023CrossGET: Cross-Guided Ensemble of Tokens for Accelerating Vision-Language Transformerscited by 44
  23. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44
  24. 2024A Survey on Transformer Compressioncited by 44
  25. 2024SlimGPT: Layer-wise Structured Pruning for Large Language Modelscited by 43
  26. 2025Weight-sparse transformers have interpretable circuitscited by 40
  27. 2024Dissecting Language Models: Machine Unlearning via Selective Pruningcited by 38
  28. 2023The LLM Surgeoncited by 36
  29. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  30. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  31. 2024Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compressioncited by 34
  32. 2024APTQ: Attention-aware Post-Training Mixed-Precision Quantization for Large Language Modelscited by 33
  33. 2023Beyond Size: How Gradients Shape Pruning Decisions in Large Language Modelscited by 30
  34. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  35. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 30
  36. 2025Bridging the Gap Between Promise and Performance for Microscaling FP4 Quantizationcited by 30
  37. 2024SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spikingcited by 29
  38. 2025Activation-Informed Merging of Large Language Modelscited by 29

50 most cited of 317 All 317 (JSON)