mlsanityData to 7 Oct 2026

CBQ: Cross-Block Quantization for Large Language Models

arXiv 2312.07950 ↗First posted 13 Dec 2023cs.LG cs.CLcited by 34

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) has played a key role in compressing large language models (LLMs) with ultra-low costs. However, existing PTQ methods only focus on handling the outliers within one layer or one block, which ignores the dependency of blocks and leads to severe performance degradation in low-bit settings. In this paper, we propose CBQ, a cross-block reconstruction-based PTQ method for LLMs. CBQ employs a cross-block dependency using a homologous reconstruction scheme, establishing long-range dependencies across multiple blocks to minimize error accumulation. Furthermore, CBQ incorporates a coarse-to-fine preprocessing (CFP) strategy for suppressing weight and activation outliers, coupled with an adaptive LoRA-Rounding technique for precise weight quantization. These innovations enable CBQ to not only handle extreme outliers effectively but also improve overall quantization accuracy. Extensive experiments show that CBQ achieves superior low-bit quantization (W4A4, W4A8, W2A16) and outperforms existing state-of-the-art methods across various LLMs and datasets. Notably, CBQ quantizes the 4-bit LLAMA1-65B model within only 4.3 hours on a single GPU, achieving a commendable tradeoff between performance and quantization efficiency.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 49 of its 70 references that are in the map. First posted on arXiv 13 Dec 2023.

Builds on

49 of 70 references.

References · 49

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  9. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  10. 2016Pointer Sentinel Mixture Modelscited by 2,992
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  12. 2022Emergent Abilities of Large Language Modelscited by 2,483
37 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  4. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  6. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  7. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  8. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  9. 2020Aligning AI With Shared Human Valuescited by 706
  10. 2017Channel Pruning for Accelerating Very Deep Neural Networkscited by 697
  11. 2019Learned Step Size Quantizationcited by 604
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  13. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  14. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  15. 2021A White Paper on Neural Network Quantizationcited by 453
  16. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  17. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  18. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  19. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  20. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  21. 2021Post-Training Quantization for Vision Transformercited by 274
  22. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  23. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  24. 2023Token Merging for Fast Stable Diffusioncited by 220
  25. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  26. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 176
  27. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  28. 2023The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Datasetcited by 148
  29. 2020MuTual: A Dataset for Multi-Turn Dialogue Reasoningcited by 114
  30. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  31. 2022Q-ViT: Accurate and Fully Quantized Low-bit Vision Transformercited by 106
  32. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 89
  33. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  34. 2020EasyQuant: Post-training Quantization via Scale Optimizationcited by 57
  35. 2023ZeroQuant-FP: A Leap Forward in LLMs Post-Training W4A8 Quantization Using Floating-Point Formatscited by 44
  36. 2017Balanced Quantization: An Effective and Efficient Approach to Quantized Neural Networkscited by 21
  37. 2018Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)cited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 15; 2026: 12.

34 papers.

Sort:

By area:

  1. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  2. 2024Q-VLM: Post-training Quantization for Large Vision-Language Modelscited by 47
  3. 2024A Survey on Transformer Compressioncited by 44
  4. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 36
  5. 2025Model-Preserving Adaptive Roundingcited by 20
  6. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysiscited by 12
  7. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  8. 2024Rethinking Pruning Large Language Models: Benefits and Pitfalls of Reconstruction Error Minimizationcited by 8
  9. 2025PTQ1.61: Push the Real Limit of Extremely Low-Bit Post-Training Quantization Methods for Large Language Modelscited by 7
  10. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 7
  11. 2026CoreQ: Learning-Free Mismatch Correction and Successive Rounding for Quantizationcited by 5
  12. 2026SliderQuant: Accurate Post-Training Quantization for LLMscited by 5
22 more
  1. 2026CAT-Q: Cost-efficient and Accurate Ternary Quantization for LLMscited by 5
  2. 2025Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compressioncited by 4
  3. 2026OneComp: One-Line Revolution for Generative AI Model Compressioncited by 3
  4. 2024How to Parameterize Asymmetric Quantization Ranges for Quantization-Aware Trainingcited by 2
  5. 2024Interactions Across Blocks in Post-Training Quantization of Large Language Modelscited by 1
  6. 2025NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMscited by 1
  7. 2025Rethinking 1-bit Optimization Leveraging Pre-trained Large Language Modelscited by 1
  8. 2025DPQuant: Efficient and Differentially-Private Model Training via Dynamic Quantization Schedulingcited by 1
  9. 2026Breaking Modality Heterogeneity in Low-Bit Quantization for Large Vision-Language Modelscited by 1
  10. 2026Why Does Post-Training Quantization Work?cited by 1
  11. 2024BiSup: Bidirectional Quantization Error Suppression for Large Language Modelscited by 0
  12. 2025Cat: Post-Training Quantization Error Reduction via Cluster-based Affine Transformationcited by 0
  13. 2025Neural Weight Compression for Language Modelscited by 0
  14. 2025Layer-Wise High-Impact Parameter Ratio Optimization in Post-Training Quantization for Large Language Modelscited by 0
  15. 2025Can abstract concepts from LLM improve SLM performance?cited by 0
  16. 2025Rethinking Output Alignment For 1-bit Post-Training Quantization of Large Language Modelscited by 0
  17. 2026Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantizationcited by 0
  18. 2026Joint Post-Training Quantization of Vision Transformers with Learned Prompt-Guided Data Generationcited by 0
  19. 2026From Sweep to Seam: Interleaved Cross-Block Post-Training Quantizationcited by 0
  20. 2026Unifying Depth and Width Pruning for LLMs via Binary Knapsack Optimizationcited by 0
  21. 2026The Asymmetric Harms of LLM Compressioncited by 0
  22. 2026The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantizationcited by 0