mlsanityData to 7 Oct 2026

Qrazor: Reliable and Effortless 4-bit LLM Quantization by Significant Data Razoring

arXiv 2501.13331 ↗First posted 23 Jan 2025cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

Large-scale language models (LLMs) excel in language processing tasks but face deployment challenges due to high memory and computational demands. While low-bit quantization, such as 4-bit techniques, offers a potential solution, these methods often suffer from significant accuracy loss or require considerable effort for implementation such as reordering, rotation, etc. To address these challenges, we propose QRazor, a simple yet effective quantization scheme that enables 4-bit quantization of weights, activations, and KV cache in transformer-based LLMs. QRazor operates in two stages: first, quantizing data using 8 or 16-bit integers as a basis with absolute max scaling to preserve accuracy close to full-precision models, and second, compressing the quantized data to 4-bit using our significant data razoring (SDR) technique, which retains only the four most salient bits. Without any additional requirment of fine-tuning or additional training, QRazor achieves performance similar or better compared to state-of-the-art in 4-bit quantization method, surpassing Smoothquant and QLLM by over 12 points and Quarot(RTN) by more than 2.9 points in zero-shot reasoning task accuracy on the LLaMA2-7B model. Additionally, we introduce an integer-based arithmetic unit optimized for QRazor, allowing direct low-precision operations on SDR data without decompression.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 25 of its 35 references that are in the map. First posted on arXiv 23 Jan 2025.

Builds on

25 of 35 references.

References · 25

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  4. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  5. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  6. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  7. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  8. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
  11. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 359
  12. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
13 more
  1. 2023Atom: Low-bit Quantization for Efficient and Accurate LLM Servingcited by 255
  2. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 200
  3. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 113
  4. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 83
  5. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  6. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  7. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 49
  8. 2023QUIK: Towards End-to-End 4-Bit Inference on Generative Large Language Modelscited by 39
  9. 2023Norm Tweaking: High-performance Low-bit Quantization of Large Language Modelscited by 33
  10. 2022The choice of scaling technique matters for classification performancecited by 25
  11. 2023Integer or Floating Point? New Outlooks for Low-Bit Quantization on Large Language Modelscited by 18
  12. 2024OutlierTune: Efficient Channel-Wise Quantization for Large Language Modelscited by 4
  13. 2022Post-Training Quantization for Energy Efficient Realization of Deep Neural Networkscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026KV Cache Optimization Strategies for Scalable and Efficient LLM Inferencecited by 3
  2. 2026Cloud to Edge: Benchmarking LLM Inference On Hardware-Accelerated Single-Board Computerscited by 1
  3. 2025CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  2. 2024MixPE: Quantization and Hardware Co-design for Efficient LLM Inference
  3. 2025MergeQuant: Accurate 4-bit Static Quantization of Large Language Models by Channel-wise Calibration
  4. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospects
  5. 2024"Give Me BF16 or Give Me Death"? Accuracy-Performance Trade-Offs in LLM Quantization
  6. 2024WKVQuant: Quantizing Weight and Key/Value Cache for Large Language Models Gains More
  7. 2024LLMC: Benchmarking Large Language Model Quantization with a Versatile Compression Toolkit
  8. 2025Cocktail: Chunk-Adaptive Mixed-Precision Quantization for Long-Context LLM Inference