Fast-TurboQuant: A Multiplier-Free Online Vector Quantization Approach
On the map: Efficient LLM inference and serving → KV cache quantization
Abstract (arXiv)
As large language models scale, memory bandwidth for key-value caches and retrieval-augmented generation systems becomes a critical bottleneck. While 1-bit quantization addresses this constraint, recent TurboQuant relies on dense random rotation matrices to condition the vector distribution before quantization. This projection demands millions of floating-point multiplications per embedding, making it difficult to deploy on constrained edge silicon. We introduce Fast-TurboQuant, a multiplier-free projection architecture that replaces the dense matrix with a structured fast Johnson-Lindenstrauss transform. By applying a Rademacher phase inversion followed by a fast Walsh-Hadamard transform (FWHT), the method leverages sub-Gaussian concentration to satisfy the prerequisites of scalar Lloyd-Max quantization without Gaussian projections. This substitution reduces the arithmetic complexity to only additions, eliminating hardware multipliers. Evaluation on DBpedia OpenAI-3 Large embeddings demonstrates a 19.7 times algorithmic speedup under sequential execution. Furthermore, the dimension expansion due to the FWHT zero-padding reduces the mean squared error and improves Recall@10.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 11 of its 19 references that are in the map. First posted on arXiv 19 Jun 2026.
Builds on
11 of 19 references.
Closest ancestors · 5
Same direction, newest first
- 2026Provable Quantization with Randomized Hadamard Transform13 May
- 2025KVLinC : KV Cache Quantization with Hadamard Rotation and Linear Correction6 Oct
- 2024PQCache: Product Quantization-based KVCache for Long Context LLM Inference1 Jul
- 2024SpinQuant: LLM quantization with learned rotations26 May
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar
Other references · 6
- 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 48
- 2024ShiftAddLLM: Accelerating Pretrained LLMs via Post-Training Multiplication-Less Reparameterizationcited by 30
- 2026Revisiting RaBitQ and TurboQuant: A Symmetric Comparison of Methods, Theory, and Experimentscited by 3
- 2026PolarQuant: Optimal Gaussian Weight Quantization via Hadamard Rotation for LLM Compressioncited by 2
- 2026ITQ3_S: High-Fidelity 3-bit LLM Inference via Interleaved Ternary Quantization with Rotation-Domain Smoothingcited by 1
- 2026GoQuant: Geometric Orthogonal Residual Projection for Multiplier-Free Power-of-Two Transformer Quantizationcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate
- 2025ButterflyQuant: Ultra-low-bit LLM Quantization through Learnable Orthogonal Butterfly Transforms
- 2026Fast Quantized Johnson-Lindenstrauss Transforms
- 2026NanoQuant: Efficient Sub-1-Bit Quantization of Large Language Models
- 2026XOR-Trellis: Ultra-Low-Complexity Dequantization and Curvature-Aware Hadamard-Free LLM Quantization
- 2025WUSH: Near-Optimal Adaptive Transforms for LLM Quantization
- 2025PolarQuant: Quantizing KV Caches with Polar Transformation
- 2025Gradual Binary Search and Dimension Expansion : A general method for activation quantization in LLMs