VQ-bench: A Composable Vector Quantization Framework
Not on the map. Why?
Abstract (arXiv)
Vector quantization is an old problem but has recently become central to AI infrastructure. It is therefore experiencing a surge of renewed engineering and research activity. This paper provides a unified framework for developing and benchmarking new quantization algorithms. We describe 7 common conceptual quantization primitives and show how to compose them arbitrarily. We then re-express 25 common quantizers as pipelines of these primitives. Finally, we publish VQ-bench as open-source to be extended further and make reproducible benchmarks publicly available.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 34 of its 69 references that are in the map. First posted on arXiv 31 Jul 2026.
Builds on
34 of 69 references.
References · 34
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
- 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
- 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
- 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2019Learned Step Size Quantizationcited by 604
- 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
- 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
- 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
- 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
22 more
- 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
- 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
- 2024SpinQuant: LLM quantization with learned rotationscited by 446
- 2023QuIP: 2-Bit Quantization of Large Language Models With Guaranteescited by 397
- 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
- 2024QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebookscited by 354
- 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
- 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 223
- 2022FP8 Formats for Deep Learningcited by 187
- 2024GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLMcited by 155
- 2023Microscaling Data Formats for Deep Learningcited by 132
- 2018ANN-Benchmarks: A Benchmarking Tool for Approximate Nearest Neighbor Algorithmscited by 130
- 2025TurboQuant: Online Vector Quantization with Near-optimal Distortion Ratecited by 116
- 2018ProxQuant: Quantized Neural Networks via Proximal Operatorscited by 56
- 2021EDEN: Communication-Efficient and Robust Distributed Mean Estimation for Federated Learningcited by 53
- 2024QJL: 1-Bit Quantized JL Transform for KV Cache Quantization with Zero Overheadcited by 47
- 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 46
- 2022Results of the NeurIPS'21 Challenge on Billion-Scale Approximate Nearest Neighbor Searchcited by 45
- 2024Results of the Big ANN: NeurIPS'23 competitioncited by 21
- 2024Practical and Asymptotically Optimal Quantization of High-Dimensional Vectors in Euclidean Space for Approximate Nearest Neighbor Searchcited by 19
- 2026Provable Quantization with Randomized Hadamard Transformcited by 6
- 2024Optimal and Near-Optimal Adaptive Vector Quantizationcited by 2
Built on by
Not cited yet.