mlsanityData to 7 Oct 2026

OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models

arXiv 2306.02272 ↗First posted 4 Jun 2023cs.CLcited by 113

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) with hundreds of billions of parameters require powerful server-grade GPUs for inference, limiting their practical deployment. To address this challenge, we introduce the outlier-aware weight quantization (OWQ) method, which aims to minimize LLM's footprint through low-precision representation. OWQ prioritizes a small subset of structured weights sensitive to quantization, storing them in high-precision, while applying highly tuned quantization to the remaining dense weights. This sensitivity-aware mixed-precision scheme reduces the quantization error notably, and extensive experiments demonstrate that 3.1-bit models using OWQ perform comparably to 4-bit models optimized by OPTQ. Furthermore, OWQ incorporates a parameter-efficient fine-tuning for task-specific adaptation, called weak column tuning (WCT), enabling accurate task-specific LLM adaptation with minimal memory overhead in the optimized format. OWQ represents a notable advancement in the flexibility, efficiency, and practicality of LLM optimization literature. The source code is available at https://github.com/xvyaward/owq

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 4 Jun 2023.

Builds on

25 of 36 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  7. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  9. 2016Pointer Sentinel Mixture Modelscited by 2,992
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,201
  11. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  12. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
13 more
  1. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  2. 2019Learned Step Size Quantizationcited by 604
  3. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  5. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  6. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  7. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  8. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 317
  9. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  10. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  11. 2022LUT-GEMM: Quantized Matrix Multiplication based on LUTs for Efficient Inference in Large-Scale Generative Language Modelscited by 98
  12. 2019Loss Aware Post-training Quantizationcited by 94
  13. 2018Value-aware Quantization for Training and Inference of Neural Networkscited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 31; 2025: 46; 2026: 34.

113 papers.

Sort:

By area:

  1. 2023A Survey on Model Compression for Large Language Modelscited by 263
  2. 2023Efficient Large Language Models: A Surveycited by 183
  3. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  4. 2024Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Modelscited by 71
  5. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 59
  6. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  7. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 53
  8. 2024The Super Weight in Large Language Modelscited by 51
  9. 2024Any-Precision LLM: Low-Cost Deployment of Multiple, Different-Sized LLMscited by 50
  10. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  11. 2024CorDA: Context-Oriented Decomposition Adaptation of Large Language Models for Task-Aware Parameter-Efficient Fine-tuningcited by 41
  12. 2024ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residualscited by 32
38 more
  1. 2024STBLLM: Breaking the 1-Bit Barrier with Structured Binary LLMscited by 31
  2. 2024ARB-LLM: Alternating Refined Binarizations for Large Language Modelscited by 30
  3. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 22
  4. 2024Spectra: Surprising Effectiveness of Pretraining Ternary Language Models at Scalecited by 20
  5. 2024QSpec: Speculative Decoding with Complementary Quantization Schemescited by 18
  6. 2024MixLLM: LLM Quantization with Global Mixed-precision between Output-features and Highly-efficient System Designcited by 17
  7. 2024Effective Interplay between Sparsity and Quantization: From Theory to Practicecited by 16
  8. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  9. 2025ParoQuant: Pairwise Rotation Quantization for Efficient Reasoning LLM Inferencecited by 16
  10. 2025Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compressioncited by 15
  11. 2024Mixture of Scales: Memory-Efficient Token-Adaptive Binarization for Large Language Modelscited by 14
  12. 2025DartQuant: Efficient Rotational Distribution Calibration for LLM Quantizationcited by 14
  13. 2025Oaken: Fast and Efficient LLM Serving with Online-Offline Hybrid KV Cache Quantizationcited by 13
  14. 2025Benchmarking Post-Training Quantization in LLMs: Comprehensive Taxonomy, Unified Evaluation, and Comparative Analysiscited by 12
  15. 2024Channel-Wise Mixed-Precision Quantization for Large Language Modelscited by 11
  16. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  17. 2025MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Modelscited by 11
  18. 2024Post Training Quantization of Large Language Models with Microscaling Formatscited by 10
  19. 2024PTQ4VM: Post-Training Quantization for Visual Mambacited by 9
  20. 2025Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMscited by 9
  21. 2025Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deploymentcited by 9
  22. 2025Fast and Low-Cost Genomic Foundation Models via Outlier Removalcited by 8
  23. 2025GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuningcited by 8
  24. 2024DecDEC: A Systems Approach to Advancing Low-Bit LLM Quantizationcited by 7
  25. 2024QUICK: Quantization-aware Interleaving and Conflict-free Kernel for efficient LLM inferencecited by 6
  26. 2025FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformerscited by 6
  27. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 6
  28. 2026Why and When Visual Token Pruning Fails? A Study on Relevant Visual Information Shift in MLLMs Decodingcited by 6
  29. 2025RoSTE: An Efficient Quantization-Aware Supervised Fine-Tuning Approach for Large Language Modelscited by 5
  30. 2025RaanA: A Fast, Flexible, and Data-Efficient Post-Training Quantization Algorithmcited by 5
  31. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Modelscited by 5
  32. 2024Scaling Laws for Post Training Quantized Large Language Modelscited by 4
  33. 2025Radio: Rate-Distortion Optimization for Large Language Model Compressioncited by 4
  34. 2025Dynamic Context-oriented Decomposition for Task-aware Low-rank Adaptation with Less Forgetting and Faster Convergencecited by 4
  35. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 4
  36. 2026GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Samplingcited by 4
  37. 2026Quantizing With Randomized Hadamard Transforms: Efficient Heuristic Now Provencited by 4
  38. 2024OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Modelscited by 3

50 most cited of 113 All 113 (JSON)