mlsanityData to 14 Oct 2026

OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models

arXiv 2308.13137 ↗First posted 25 Aug 2023cs.LG cs.CLcited by 424

Not on the map. Why?

Cited by 36 in Efficient LLM inference and serving, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have revolutionized natural language processing tasks. However, their practical deployment is hindered by their immense memory and computation requirements. Although recent post-training quantization (PTQ) methods are effective in reducing memory footprint and improving the computational efficiency of LLM, they hand-craft quantization parameters, leading to low performance, especially in extremely low-bit quantization. To tackle this issue, we introduce an Omnidirectionally calibrated Quantization (OmniQuant) technique for LLMs, which achieves good performance in diverse quantization settings while maintaining the computational efficiency of PTQ by efficiently optimizing various quantization parameters. OmniQuant comprises two innovative components including Learnable Weight Clipping (LWC) and Learnable Equivalent Transformation (LET). LWC modulates the extreme values of weights by optimizing the clipping threshold. Meanwhile, LET tackles activation outliers by shifting the challenge of quantization from activations to weights. Operating within a differentiable framework using block-wise error minimization, OmniQuant can optimize the quantization process efficiently for both weight-only and weight-activation quantization. For instance, the LLaMA-2 model family size 7-70B can be processed with OmniQuant on a single A100-40G GPU within 1-16 hours using 128 samples. Extensive experiments validate OmniQuant's superior performance across diverse quantization configurations such as W4A4 (4-bit weight, 4-bit activation), W6A6, W4A16, W3A16, and W2A16. Additionally, OmniQuant demonstrates effectiveness in instruction-tuned models and delivers notable improvements in inference speed and memory reduction on real devices. Codes are available at urlhttps://github.com/OpenGVLab/OmniQuant.

Key facts. Cited by 424 papers in the map (data through 14 Oct 2026). Builds on 40 of its 51 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

40 of 51 references.

References · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  11. 2016Pointer Sentinel Mixture Modelscited by 3,096
  12. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
28 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  4. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  8. 2023The RefinedWeb Dataset for Falcon LLM: Outperforming Curated Corpora with Web Data, and Web Data Onlycited by 763
  9. 2019Learned Step Size Quantizationcited by 628
  10. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  11. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  12. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  13. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  14. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  15. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  16. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  17. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  18. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 279
  19. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  20. 2023LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Modelscited by 217
  21. 2022Outlier Suppression: Pushing the Limit of Low-bit Transformer Language Modelscited by 181
  22. 2022RepQ-ViT: Scale Reparameterization for Post-Training Quantization of Vision Transformerscited by 137
  23. 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
  24. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 116
  25. 2021Nonuniform-to-Uniform Quantization: Towards Accurate Quantization via Generalized Straight-Through Estimationcited by 79
  26. 2023You Only Segment Once: Towards Real-Time Panoptic Segmentationcited by 40
  27. 2023INT2.1: Towards Fine-Tunable Quantized Large Language Models with Error Correction through Low-Rank Adaptationcited by 20
  28. 2023Geometric-aware Pretraining for Vision-centric 3D Object Detectioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 12; 2024: 113; 2025: 127; 2026: 172.

424 papers.

Sort:

By area:

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  2. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 591
  3. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  4. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  5. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  6. 2024A Survey on Efficient Inference for Large Language Modelscited by 205
  7. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  8. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  9. 2024A Survey on Large Language Model Acceleration based on KV Cache Managementcited by 131
  10. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 128
  11. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  12. 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumptioncited by 97
38 more
  1. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  2. 2023LQ-LoRA: Low-rank Plus Quantized Matrix Decomposition for Efficient Language Model Finetuningcited by 92
  3. 2024QTIP: Quantization with Trellises and Incoherence Processingcited by 91
  4. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 86
  5. 2025DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMscited by 84
  6. 2023PB-LLM: Partially Binarized Large Language Modelscited by 83
  7. 2024GPTVQ: The Blessing of Dimensionality for LLM Quantizationcited by 82
  8. 2024OneBit: Towards Extremely Low-bit Large Language Modelscited by 79
  9. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 79
  10. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  11. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 64
  12. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 64
  13. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMscited by 64
  14. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  15. 2023Dynamic Sparse No Training: Training-Free Fine-tuning for Sparse LLMscited by 61
  16. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  17. 2025GPTAQ: Efficient Finetuning-Free Quantization for Asymmetric Calibrationcited by 60
  18. 2024Training-Free Activation Sparsity in Large Language Modelscited by 55
  19. 2024The Super Weight in Large Language Modelscited by 52
  20. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  21. 2024A Survey on Transformer Compressioncited by 46
  22. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 46
  23. 2024BESA: Pruning Large Language Models with Blockwise Parameter-Efficient Sparsity Allocationcited by 45
  24. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Modelscited by 45
  25. 2024SKVQ: Sliding-window Key and Value Cache Quantization for Large Language Modelscited by 44
  26. 2024Low-Rank Quantization-Aware Training for LLMscited by 43
  27. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 41
  28. 2024Accurate LoRA-Finetuning Quantization of LLMs via Information Retentioncited by 41
  29. 2024Mixture Compressor for Mixture-of-Experts LLMs Gains Morecited by 41
  30. 2025Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantizationcited by 41
  31. 2023QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Modelscited by 39
  32. 2023SiDA-MoE: Sparsity-Inspired Data-Aware Serving for Efficient and Scalable Large Mixture-of-Experts Modelscited by 38
  33. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 36
  34. 2023CBQ: Cross-Block Quantization for Large Language Modelscited by 34
  35. 2024SpikeLLM: Scaling up Spiking Neural Network to Large Language Models via Saliency-based Spikingcited by 31
  36. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  37. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 31
  38. 20241.58-bit FLUXcited by 31

50 most cited of 424 All 424 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
  2. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Models
  3. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models
  4. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation
  5. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  6. 2023FPTQ: Fine-grained Post-Training Quantization for Large Language Models
  7. 2026RUQuant: Towards Refining Uniform Quantization for Large Language Models
  8. 2023Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization