# FPTQuant: Function-Preserving Transforms for LLM Quantization

arXiv: https://arxiv.org/abs/2506.04985 · HTML: https://mlsanity.com/paper/2506.04985/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 78 references that are in the map.
- First posted on arXiv 5 Jun 2025.

## Builds on (most cited first, 30 of 53)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2211.10438/) (2022)
- [The LAMBADA dataset: Word prediction requiring a broad discourse context](https://mlsanity.com/paper/1606.06031/) (2016)
- [QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs](https://mlsanity.com/paper/2404.00456/) (2024)
- [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [A White Paper on Neural Network Quantization](https://mlsanity.com/paper/2106.08295/) (2021)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [QuIP: 2-Bit Quantization of Large Language Models With Guarantees](https://mlsanity.com/paper/2307.13304/) (2023)
- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://mlsanity.com/paper/2401.15024/) (2024)
- [QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks](https://mlsanity.com/paper/2402.04396/) (2024)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression](https://mlsanity.com/paper/2306.03078/) (2023)
- [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019)
- [LLM-QAT: Data-Free Quantization Aware Training for Large Language Models](https://mlsanity.com/paper/2305.17888/) (2023)
- [SqueezeLLM: Dense-and-Sparse Quantization](https://mlsanity.com/paper/2306.07629/) (2023)
- [Massive Activations in Large Language Models](https://mlsanity.com/paper/2402.17762/) (2024)

## Built on by (most cited first, 16 of 16)

- [Block Rotation is All You Need for MXFP4 Quantization](https://mlsanity.com/paper/2511.04214/) (2025)
- [Leech Lattice Vector Quantization for Efficient LLM Compression](https://mlsanity.com/paper/2603.11021/) (2026)
- [Dissecting Quantization Error: A Concentration-Alignment Perspective](https://mlsanity.com/paper/2603.04359/) (2026)
- [StableQAT: Stable Quantization-Aware Training at Ultra-Low Bitwidths](https://mlsanity.com/paper/2601.19320/) (2026)
- [Neural Weight Compression for Language Models](https://mlsanity.com/paper/2510.11234/) (2025)
- [STaMP: Sequence Transformation and Mixed Precision for Low-Precision Activation Quantization](https://mlsanity.com/paper/2510.26771/) (2025)
- [CafeQ: Calibration-free Quantization via Learned Transformations and Adaptive Rounding](https://mlsanity.com/paper/2511.19705/) (2025)
- [A Case Study of Selected PTQ Baselines for Reasoning LLMs on Ascend NPU](https://mlsanity.com/paper/2602.17693/) (2026)
- [FAQ: Mitigating Quantization Error via Regenerating Calibration Data with Family-Aware Quantization](https://mlsanity.com/paper/2601.11200/) (2026)
- [Efficient Reasoning on the Edge](https://mlsanity.com/paper/2603.16867/) (2026)
- [When Quantization Is Free: An int4 KV Cache That Outruns fp16 on Apple Silicon](https://mlsanity.com/paper/2605.05699/) (2026)
- [RotateAttention: RoPE-Aware Rotation and Range Rectification for INT4 Quantized Attention in Video Generation](https://mlsanity.com/paper/2607.02584/) (2026)
- [When Local Variance Optimality Is Not Enough: RoPE-Aligned Q/K Rotations for Dynamic 4-Bit Quantisation](https://mlsanity.com/paper/2608.13365/) (2026)
- [G2PTQ: Improving LLM Post-Training Quantization with Generalized Gradient Compensation](https://mlsanity.com/paper/2609.31009/) (2026)
- [QuantMLA: Function-Aligned Dual-Path Quantization for Low-Bit MLA KV Caching](https://mlsanity.com/paper/2609.36760/) (2026)
- [The Devil Is in the Reconstruction Loss Scale: Rethinking Optimization in LLM Quantization](https://mlsanity.com/paper/2610.00983/) (2026)

## Abstract (arXiv)

Large language models (LLMs) require substantial compute, and thus energy, at inference time. While quantizing weights and activations is effective at improving efficiency, naive quantization of LLMs can significantly degrade performance due to large magnitude outliers. This paper describes FPTQuant, which introduces three novel, lightweight, and expressive function-preserving transforms (FPTs) to facilitate quantization of transformers: (1) a mergeable pre-RoPE transform for queries and keys, (2) a mergeable transform for values, and (3) a cheap, dynamic per-token scaling transform. By leveraging the equivariances and independencies inherent to canonical transformer operation, we designed these FPTs to maintain the model's function while shaping the intermediate activation distributions to be more quantization friendly. FPTQuant requires no custom kernels and adds virtually no overhead during inference. The FPTs are trained both locally to reduce outliers, and end-to-end such that the outputs of the quantized and full-precision models match. FPTQuant enables static INT4 quantization with minimal overhead and shows SOTA speed-up of up to 3.9X over FP. Empirically, FPTQuant has an excellent accuracy-speed trade-off -- it is performing on par or exceeding most prior work and only shows slightly lower accuracy compared to a method that is up to 29% slower.