When Does Low-Bit Quantization Preserve the Decisions of Vector Search?
Not on the map. Why?
Abstract (arXiv)
Low-bit quantization can achieve high recall on some vector representations and fail sharply on others, while average distortion and global rank correlation do not explain the difference. We study quantized vector search at the level of the comparisons consumed by ranking and graph-pruning algorithms. Our first result is a distribution-free decomposition: the probability that a comparison flips is bounded by the probability mass of exact margins near zero plus the tail probability of the calibrated residual. We then account for dependence between residuals that share a query or graph node, and derive covariance-aware second-moment identities and tail bounds under a joint MGF proxy. For a frozen candidate permutation, we prove a deterministic coupling theorem for Vamana neighbour selection: the approximate replay returns the exact neighbour list exactly when all candidate-level pruning actions agree on the frozen exact states. We connect these results to representation geometry through an exact Gaussian oracle, establish a strict correlation gain from a deterministic magnitude bit in an aligned bilinear model, and give a rare-contamination construction showing why marginal Gaussian diagnostics do not imply the required residual tails. When analytical assumptions are unavailable, a held-out block certificate bounds the selective failure risk of a frozen quantized rule. Across learned, classical, and synthetic embeddings, standardized exact margins predict held-out ranking and pruning flip rates substantially better than global rank correlation. The framework applies to coordinate binary codes, RaBitQ, Lucene BBQ, and product quantizers through a common decision interface.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 4 of its 15 references that are in the map. First posted on arXiv 9 Sep 2026.
Builds on
4 of 15 references.
References · 4
- 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 861
- 2024RaBitQ: Quantizing High-Dimensional Vectors with a Theoretical Error Bound for Approximate Nearest Neighbor Searchcited by 48
- 2026InfoNCE Induces Gaussian Distributioncited by 16
- 2026QuIVer: Rethinking ANN Graph Topology via Training-Free Binary Quantizationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026RSLM: Training-Free Vector Quantization for Approximate Nearest Neighbor Search
- 2026Covariance Structure and Coordinate Heterogeneity Govern Binary Quantization of Contrastive Embeddings
- 2026MRVQ: One Resident Index for Dimension- and Rate-Elastic Vector Search
- 2025Graph is a Natural Regularization: Revisiting Vector Quantization for Graph Representation Learning
- 2026Where Post-Training Quantization Breaks Text Embedders: A Measured Map Across Four Embedder Families
- 2026Zero-Shot Quantization via Weight-Space Arithmetic
- 2025Individualized non-uniform quantization for vector search
- 2023Similarity search in the blink of an eye with compressed indices