mlsanityData to 7 Oct 2026

Residual Quantization with Implicit Neural Codebooks

arXiv 2401.14732 ↗First posted 26 Jan 2024cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Vector quantization is a fundamental operation for data compression and vector search. To obtain high accuracy, multi-codebook methods represent each vector using codewords across several codebooks. Residual quantization (RQ) is one such method, which iteratively quantizes the error of the previous step. While the error distribution is dependent on previously-selected codewords, this dependency is not accounted for in conventional RQ as it uses a fixed codebook per quantization step. In this paper, we propose QINCo, a neural RQ variant that constructs specialized codebooks per step that depend on the approximation of the vector from previous steps. Experiments show that QINCo outperforms state-of-the-art methods by a large margin on several datasets and code sizes. For example, QINCo achieves better nearest-neighbor search accuracy using 12-byte codes than the state-of-the-art UNQ using 16 bytes on the BigANN1M and Deep1M datasets.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 35 of its 59 references that are in the map. First posted on arXiv 26 Jan 2024.

Builds on

35 of 59 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  8. 2021Unsupervised Dense Information Retrieval with Contrastive Learningcited by 1,203
  9. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  11. 2016Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphscited by 853
  12. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
23 more
  1. 2022High Fidelity Neural Audio Compressioncited by 673
  2. 2016Learning to learn by gradient descent by gradient descentcited by 616
  3. 2023Simple and Controllable Music Generationcited by 480
  4. 2023High-Fidelity Audio Compression with Improved RVQGANcited by 423
  5. 2024The Faiss librarycited by 370
  6. 2022A Self-Supervised Descriptor for Image Copy Detectioncited by 175
  7. 2017Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representationscited by 149
  8. 2021A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learningcited by 82
  9. 2023Straightening Out the Straight-Through Estimator: Overcoming Optimization Challenges in Vector Quantized Networkscited by 80
  10. 2017Learning Joint Multilingual Sentence Representations with Neural Machine Translationcited by 52
  11. 2020WeightNet: Revisiting the Design Space of Weight Networkscited by 50
  12. 2022Results of the NeurIPS'21 Challenge on Billion-Scale Approximate Nearest Neighbor Searchcited by 45
  13. 2018Revisiting the Inverted Indices for Billion-Scale Approximate Nearest Neighborscited by 37
  14. 2022Image Compression with Product Quantized Masked Image Modelingcited by 32
  15. 2021Self-supervised Product Quantization for Deep Unsupervised Image Retrievalcited by 28
  16. 2023Towards Universal Image Embeddings: A Large-Scale Dataset and Challenge for Generic Image Representationscited by 28
  17. 2017End-to-End Supervised Product Quantization for Image Search and Retrievalcited by 16
  18. 2019Deep Triplet Quantizationcited by 16
  19. 2021Contrastive Quantization with Code Memory for Unsupervised Image Retrievalcited by 16
  20. 2019Unsupervised Neural Quantization for Compressed-Domain Similarity Searchcited by 15
  21. 2016Polysemous codescited by 8
  22. 2021Semi-supervised Network Embedding with Differentiable Deep Quantisationcited by 7
  23. 2021Nearest neighbor search with compact codes: A decoder perspectivecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 8; 2026: 9.

24 papers.

Sort:

By area:

  1. 2024WavChat: A Survey of Spoken Dialogue Modelscited by 88
  2. 2024Representation Collapsing Problems in Vector Quantizationcited by 10
  3. 2025Qinco2: Vector Compression and Search with Improved Implicit Neural Codebookscited by 7
  4. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 7
  5. 2024DM-Codec: Distilling Multimodal Representations for Speech Tokenizationcited by 5
  6. 2024Balance of Number of Embedding and their Dimensions in Vector Quantizationcited by 4
  7. 2025CARVQ: Corrective Adaptor with Group Residual Vector Quantization for LLM Embedding Compressioncited by 3
  8. 2024Rate-Adaptive Quantization: A Multi-Rate Codebook Adaptation for Vector Quantization-based Generative Modelscited by 2
  9. 2026EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecturecited by 2
  10. 2024Theoretical and Empirical Analysis of Adaptive Entry Point Selection for Graph-based Approximate Nearest Neighbor Searchcited by 1
  11. 2024Random Cycle Coding: Lossless Compression of Cluster Assignments via Bits-Back Codingcited by 1
  12. 2025Lossless Compression of Vector IDs for Approximate Nearest Neighbor Searchcited by 1
12 more
  1. 2025Machine learning and high dimensional vector searchcited by 1
  2. 2026Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenizationcited by 1
  3. 2026RQ-MoE: Residual Quantization via Mixture of Experts for Efficient Input-Dependent Vector Compressioncited by 1
  4. 2025SPARC: Soft Probabilistic Adaptive multi-interest Retrieval Model via Codebooks for recommender systemcited by 0
  5. 2025Image-Conditioned 3D Gaussian Splat Quantizationcited by 0
  6. 2025PLASMA: A Layout-Aware Benchmark Reveals Memory Layout Matters for Graph-based ANNS on GPUcited by 0
  7. 2026Multi-Bitwidth Quantization for LLMs Using Additive Codebookscited by 0
  8. 2026HERMES: A Multi-Granularity Labeling Substrate for Pre-training Data Mixturescited by 0
  9. 2026Rethinking Language Model-Based Generative Speech Enhancement in the Latent Space of a Neural Audio Codeccited by 0
  10. 2026Towards a Densing Law for User Representation Learning at Billion-Scale Capacitycited by 0
  11. 2026Vortex: Bridging Extreme Compression and Efficient LLM Inferencecited by 0
  12. 2026What Makes a Good Medical Image Tokenizer? Rethinking Reconstruction and Generation in Medical Image Tokenizationcited by 0