mlsanityData to 14 Oct 2026

FPTQuant: Function-Preserving Transforms for LLM Quantization

arXiv 2506.04985 ↗First posted 5 Jun 2025cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) require substantial compute, and thus energy, at inference time. While quantizing weights and activations is effective at improving efficiency, naive quantization of LLMs can significantly degrade performance due to large magnitude outliers. This paper describes FPTQuant, which introduces three novel, lightweight, and expressive function-preserving transforms (FPTs) to facilitate quantization of transformers: (1) a mergeable pre-RoPE transform for queries and keys, (2) a mergeable transform for values, and (3) a cheap, dynamic per-token scaling transform. By leveraging the equivariances and independencies inherent to canonical transformer operation, we designed these FPTs to maintain the model's function while shaping the intermediate activation distributions to be more quantization friendly. FPTQuant requires no custom kernels and adds virtually no overhead during inference. The FPTs are trained both locally to reduce outliers, and end-to-end such that the outputs of the quantized and full-precision models match. FPTQuant enables static INT4 quantization with minimal overhead and shows SOTA speed-up of up to 3.9X over FP. Empirically, FPTQuant has an excellent accuracy-speed trade-off -- it is performing on par or exceeding most prior work and only shows slightly lower accuracy compared to a method that is up to 29% slower.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 53 of its 78 references that are in the map. First posted on arXiv 5 Jun 2025.

Builds on

53 of 78 references.

References · 53

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  5. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  6. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2016Pointer Sentinel Mixture Modelscited by 3,096
  9. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,883
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  12. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
38 more
  1. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 992
  2. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 634
  3. 2019Learned Step Size Quantizationcited by 628
  4. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  5. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  6. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  7. 2021A White Paper on Neural Network Quantizationcited by 472
  8. 2024SpinQuant: LLM quantization with learned rotationscited by 471
  9. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 424
  10. 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 414
  11. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  12. 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 378
  13. 2015Deep Learning with Limited Numerical Precisioncited by 376
  14. 2023SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compressioncited by 367
  15. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  16. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  17. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 294
  18. 2024Massive Activations in Large Language Modelscited by 294
  19. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 269
  20. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  21. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 193
  22. 2020LSQ+: Improving low-bit quantization through learnable offsets and better initializationcited by 189
  23. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  24. 2023QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Modelscited by 155
  25. 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 151
  26. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  27. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  28. 2024FlatQuant: Flatness Matters for LLM Quantizationcited by 131
  29. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 127
  30. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  31. 2023OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Modelscited by 114
  32. 2021BERT Busters: Outlier Dimensions that Disrupt Transformerscited by 112
  33. 2025OstQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fittingcited by 97
  34. 2020Improving Post Training Neural Quantization: Layer-wise Calibration and Integer Programmingcited by 92
  35. 2024BitDistiller: Unleashing the Potential of Sub-4-Bit LLMs via Self-Distillationcited by 84
  36. 2024SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Modelscited by 63
  37. 2023FlexRound: Learnable Rounding based on Element-wise Division for Post-Training Quantizationcited by 48
  38. 2019Same, Same But Different - Recovering Neural Network Quantization Error Through Weight Factorizationcited by 45

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 12.

16 papers.

Sort:

By area:

  1. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 20
  2. 2026Leech Lattice Vector Quantization for Efficient LLM Compressioncited by 7
  3. 2026Dissecting Quantization Error: A Concentration-Alignment Perspectivecited by 5
  4. 2026StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidthscited by 2
  5. 2025Neural Weight Compression for Language Modelscited by 1
  6. 2025STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantizationcited by 1
  7. 2025CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Roundingcited by 1
  8. 2026A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPUcited by 1
  9. 2026FAQ: Mitigating Quantization Error via Regenerating Calibration Data with Family-Aware Quantizationcited by 0
  10. 2026Efficient Reasoning on the Edgecited by 0
  11. 2026When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Siliconcited by 0
  12. 2026RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generationcited by 0
4 more
  1. 2026When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisationcited by 0
  2. 2026G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensationcited by 0
  3. 2026QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Cachingcited by 0
  4. 2026The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024DFRot: Achieving Outlier-Free and Massive Activation-Free for Rotated LLMs with Refined Rotation
  2. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformers
  3. 2023SmoothQuant+: Accurate and Efficient 4-bit Post-Training WeightQuantization for LLM
  4. 2023Outlier Suppression+: Accurate quantization of large language models by equivalent and optimal shifting and scaling
  5. 2023FineQuant: Unlocking Efficiency with Fine-Grained Weight-Only Quantization for LLMs
  6. 2024EasyQuant: An Efficient Data-free Quantization Algorithm for LLMs
  7. 2023TEQ: Trainable Equivalent Transformation for Quantization of LLMs
  8. 2025FlexQuant: A Flexible and Efficient Dynamic Precision Switching Framework for LLM Quantization