mlsanityData to 7 Oct 2026

FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantization

arXiv 2306.00317 ↗First posted 1 Jun 2023cs.LG cs.AIcited by 46

Not on the map. Why?

Abstract (arXiv)

Post-training quantization (PTQ) has been gaining popularity for the deployment of deep neural networks on resource-limited devices since unlike quantization-aware training, neither a full training dataset nor end-to-end training is required at all. As PTQ schemes based on reconstructing each layer or block output turn out to be effective to enhance quantized model performance, recent works have developed algorithms to devise and learn a new weight-rounding scheme so as to better reconstruct each layer or block output. In this work, we propose a simple yet effective new weight-rounding mechanism for PTQ, coined FlexRound, based on element-wise division instead of typical element-wise addition such that FlexRound enables jointly learning a common quantization grid size as well as a different scale for each pre-trained weight. Thanks to the reciprocal rule of derivatives induced by element-wise division, FlexRound is inherently able to exploit pre-trained weights when updating their corresponding scales, and thus, flexibly quantize pre-trained weights depending on their magnitudes. We empirically validate the efficacy of FlexRound on a wide range of models and tasks. To the best of our knowledge, our work is the first to carry out comprehensive experiments on not only image classification and natural language understanding but also natural language generation. Moreover, we demonstrate, for the first time, that large language models can be efficiently quantized, with only a negligible impact on performance compared to half-precision baselines, achieved by reconstructing the output in a block-by-block manner. Our code is available at urlhttps://github.com/onliwad101/FlexRoundLRQ.

Key facts. Cited by 46 papers in the map (data through 7 Oct 2026). Builds on 40 of its 59 references that are in the map. First posted on arXiv 1 Jun 2023.

Builds on

40 of 59 references.

References · 40

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  7. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  8. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  10. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,195
  11. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  12. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
28 more
  1. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  2. 2016Pointer Sentinel Mixture Modelscited by 2,992
  3. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  4. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  5. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  6. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  7. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  8. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  10. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  11. 2019Learned Step Size Quantizationcited by 604
  12. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  13. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  14. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  15. 2021A White Paper on Neural Network Quantizationcited by 453
  16. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
  17. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  18. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  19. 2022QDrop: Randomly Dropping Quantization for Extremely Low-bit Post-Training Quantizationcited by 175
  20. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  21. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
  22. 2020TernaryBERT: Distillation-aware Ultra-low Bit BERTcited by 145
  23. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 139
  24. 2019Loss Aware Post-training Quantizationcited by 94
  25. 2019Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networkscited by 87
  26. 2019AutoQ: Automated Kernel-Wise Neural Network Quantizationcited by 60
  27. 2021Towards Efficient Post-training Quantization of Pre-trained Language Modelscited by 37
  28. 2021Cluster-Promoting Quantization with Bit-Drop for Minimizing Network Quantization Losscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 19; 2025: 13; 2026: 11.

46 papers.

Sort:

By area:

  1. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 84
  3. 2024AffineQuant: Affine Transformation Quantization for Large Language Modelscited by 64
  4. 2024Model Compression and Efficient Inference for Large Language Models: A Surveycited by 62
  5. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 49
  6. 2024Low-Rank Quantization-Aware Training for LLMscited by 42
  7. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 39
  8. 2024PTQ4SAM: Post-Training Quantization for Segment Anythingcited by 32
  9. 2025Peri-LN: Revisiting Normalization Layer in the Transformer Architecturecited by 32
  10. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  11. 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 27
  12. 2023Quantization Variation: A New Perspective on Training Transformers with Low-Bit Precisioncited by 17
34 more
  1. 2024MobileQuant: Mobile-friendly Quantization for On-device Language Modelscited by 16
  2. 2025FPTQuant: Function-Preserving Transforms for LLM Quantizationcited by 16
  3. 2024Nearest is Not Dearest: Towards Practical Defense against Quantization-conditioned Backdoor Attackscited by 14
  4. 2024COMQ: A Backpropagation-Free Algorithm for Post-Training Quantizationcited by 12
  5. 2024LoQT: Low-Rank Adapters for Quantized Pretrainingcited by 12
  6. 2024LRQ: Optimizing Post-Training Quantization for Large Language Models by Learning Low-Rank Weight-Scaling Matricescited by 11
  7. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  8. 2024RILQ: Rank-Insensitive LoRA-based Quantization Error Compensation for Boosting 2-bit Large Language Model Accuracycited by 9
  9. 2025Unifying Block-wise PTQ and Distillation-based QAT for Progressive Quantization toward 2-bit Instruction-Tuned LLMscited by 7
  10. 2024Panacea: Novel DNN Accelerator using Accuracy-Preserving Asymmetric Quantization and Energy-Saving Bit-Slice Sparsitycited by 5
  11. 2025Unifying Uniform and Binary-coding Quantization for Accurate Compression of Large Language Modelscited by 5
  12. 2025Mix-QSAM: Mixed-Precision Quantization of the Segment Anything Modelcited by 4
  13. 2025Mixed-Precision Quantization for Language Models: Techniques and Prospectscited by 4
  14. 2025CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMscited by 3
  15. 2023Efficient and Robust Quantization-aware Training via Adaptive Coreset Selectioncited by 2
  16. 2025Post-Training Quantization for Diffusion Transformer via Hierarchical Timestep Groupingcited by 2
  17. 2025Accurate Sublayer Pruning for Large Language Models by Exploiting Latency and Tunability Informationcited by 2
  18. 2025Explaining How Quantization Disparately Skews a Modelcited by 2
  19. 2026LFQ: Logit-aware Final-block Quantization for Boosting the Generation Quality of Low-Bit Quantized LLMscited by 2
  20. 2024Minimize Quantization Output Error with Bias Compensationcited by 1
  21. 2024Ex Uno Pluria: Insights on Ensembling in Low Precision Number Systemscited by 1
  22. 2025Enhancing Generalization in Data-free Quantization via Mixup-class Promptingcited by 1
  23. 2026SEPTQ: A Simple and Effective Post-Training Quantization Paradigm for Large Language Modelscited by 1
  24. 2024Not All Attention is Needed: Parameter and Computation Efficient Transfer Learning for Multi-modal Large Language Modelscited by 0
  25. 2025SignRoundV2: Toward Closing the Performance Gap in Extremely Low-Bit Post-Training Quantization for LLMscited by 0
  26. 2026FAQ: Mitigating Quantization Error via Regenerating Calibration Data with Family-Aware Quantizationcited by 0
  27. 2026Revisiting Adaptive Rounding with Vectorized Reparameterization for LLM Quantizationcited by 0
  28. 2026LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMscited by 0
  29. 2026Big2Small: A Unifying Neural Network Framework for Model Compressioncited by 0
  30. 2026OffQ: Taming Structured Outliers in LLM Quantization by Offsettingcited by 0
  31. 2026Shift-and-Sum Quantization for Visual Autoregressive Modelscited by 0
  32. 2026ReQuant: Fixed-Grid Discrete Refinement for Post-Training Quantizationcited by 0
  33. 2026ReRound: Reconstructive Rounding to Resolve Midpoint Ambiguity in Calibration-Free LLM Quantizationcited by 0
  34. 2026FluxBin: Flexible LUT-based Ultra-low-bit LLM Inference by Algorithm-Kernel Synergycited by 0