mlsanityData to 14 Oct 2026

MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization

arXiv 2406.00800 ↗First posted 2 Jun 2024cs.LG cs.AIcited by 28

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a simple optimization-based preprocessing technique called Weight Magnitude Reduction (MagR) to improve the performance of post-training quantization. For each linear layer, we adjust the pre-trained floating-point weights by solving an ℓ_∞-regularized optimization problem. This process greatly diminishes the maximum magnitude of the weights and smooths out outliers, while preserving the layer's output. The preprocessed weights are centered more towards zero, which facilitates the subsequent quantization process. To implement MagR, we address the ℓ_∞-regularization by employing an efficient proximal gradient descent algorithm. Unlike existing preprocessing methods that involve linear transformations and subsequent post-processing steps, which can introduce significant overhead at inference time, MagR functions as a non-linear transformation, eliminating the need for any additional post-processing. This ensures that MagR introduces no overhead whatsoever during inference. Our experiments demonstrate that MagR achieves state-of-the-art performance on the Llama family of models. For example, we achieve a Wikitext2 perplexity of 5.95 on the LLaMA2-70B model for per-channel INT2 weight quantization without incurring any inference overhead.

Key facts. Cited by 28 papers in the map (data through 14 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 2 Jun 2024.

Builds on

40 of 56 references.

References · 40

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  8. 2021Convex optimizationcited by 2,515
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
28 more
  1. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  2. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  3. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  4. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  5. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  6. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  7. 2022DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scalecited by 407
  8. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  9. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  10. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  11. 2019Understanding Straight-Through Estimator in Training Activation Quantized Neural Netscited by 231
  12. 2016Ternary Weight Networkscited by 204
  13. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 180
  14. 2017Deep Learning with Low Precision by Half-wave Gaussian Quantizationcited by 117
  15. 2024PTQ4DiT: Post-training Quantization for Diffusion Transformerscited by 79
  16. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 67
  17. 2024QuEST: Low-bit Diffusion Model Quantization via Efficient Selective Finetuningcited by 49
  18. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33
  19. 2023QuantEase: Optimization-based Quantization for Language Modelscited by 29
  20. 2018BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weightscited by 24
  21. 2018Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networkscited by 21
  22. 2024FrameQuant: Flexible Low-Bit Quantization for Transformerscited by 18
  23. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantizationcited by 12
  24. 2016Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detectioncited by 8
  25. 2022A simple approach for quantizing neural networkscited by 8
  26. 2023Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Datacited by 5
  27. 2023R2 Loss: Range Restriction Loss for Model Compression and Quantizationcited by 3
  28. 2024Frame Quantization of Neural Networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 15; 2026: 13.

28 papers.

Sort:

By area:

  1. 2025Qronos: Correcting the Past by Shaping the Future... in Post-Training Quantizationcited by 28
  2. 2025Provable Post-Training Quantization: Theoretical Analysis of OPTQ and Qronoscited by 16
  3. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  4. 2026Pushing the Limits of Block Rotations in Post-Training Quantizationcited by 11
  5. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 7
  6. 2025FP4DiT: Towards Effective Floating Point Quantization for Diffusion Transformerscited by 6
  7. 2025CLoQ: Enhancing Fine-Tuning of Quantized LLMs via Calibrated LoRA Initializationcited by 5
  8. 2025LoaQ: Layer-wise Output Approximation Quantizationcited by 5
  9. 2025CondiQuant: Condition Number Based Low-Bit Quantization for Image Super-Resolutioncited by 4
  10. 2026FreeAct: Freeing Activations for LLM Quantizationcited by 4
  11. 2025Symmetric Pruning of Large Language Modelscited by 3
  12. 2025Beacon: Post-Training Quantization with Integrated Grid Selectioncited by 3
16 more
  1. 2026Forget by Uncertainty: Orthogonal Entropy Unlearning for Quantized Neural Networkscited by 3
  2. 2025DiaBlo: Diagonal Blocks Are Sufficient For Finetuningcited by 2
  3. 2026OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantizationcited by 2
  4. 2026Contraction-Gauge Preconditioning for Quantized Matrix Multiplicationcited by 2
  5. 2025NeUQI: Near-Optimal Uniform Quantization Parameter Initialization for Low-Bit LLMscited by 1
  6. 2025Q-SAM2: Accurate Quantization for Segment Anything Model 2cited by 1
  7. 2025OptRot: Mitigating Weight Outliers via Data-Free Rotations for Post-Training Quantizationcited by 1
  8. 2026GPTQ-intrinsic LoRA: A Near-optimal Algorithm for Low-precision Quantization with Low-rank Adaptationcited by 1
  9. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 1
  10. 2026ZO-Act: Efficient Zeroth-Order Fine-Tuning via One-Shot Activation-Informed Low-Rank Subspacescited by 1
  11. 2025SIRA: Scaled-Integer Range Analysis for Optimizing FPGA Dataflow Neural Network Acceleratorscited by 0
  12. 2026Astro: Activation-guided Structured Regularization for Outlier-Robust LLM Post-Training Quantizationcited by 0
  13. 2026QTALE: Quantization-Robust Token-Adaptive Layer Execution for LLMscited by 0
  14. 2026Weight Group-wise Post-Training Quantization for Medical Foundation Modelcited by 0
  15. 2026Toward Multi-Domain and Long-Tailed Quantization via Feature Alignment and Scalingcited by 0
  16. 2026Lost in the bf16 Cast: Exporting Ternary Language Models Can Revert Most Low-Learning-Rate Code Changescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Identifying Sensitive Weights via Post-quantization Integral
  2. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matrices
  3. 2024Understanding the Difficulty of Low-Precision Post-Training Quantization for LLMs
  4. 2021Q-Rater: Non-Convex Optimization for Post-Training Uniform Quantization
  5. 2026Quantization-Aware Pre-Training with Constrained Empirical Weight Distribution
  6. 2025Improving Quantization with Post-Training Model Expansion
  7. 2026MARR: Module-Adaptive Residual Reconstruction for Low-Bit Post-Training Quantization
  8. 2023Gradient-Based Post-Training Quantization: Challenging the Status Quo