# Differentiable Product Quantization for End-to-End Embedding Compression

arXiv: https://arxiv.org/abs/1908.09756 · HTML: https://mlsanity.com/paper/1908.09756/

## Key facts

- Cited by 36 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 26 Aug 2019.

## Built on by (most cited first, 30 of 36)

- [Finite Scalar Quantization: VQ-VAE Made Simple](https://mlsanity.com/paper/2309.15505/) (2023)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders](https://mlsanity.com/paper/2210.12316/) (2022)
- [VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models](https://mlsanity.com/paper/2409.17066/) (2024)
- [FoldToken: Learning Protein Language via Vector Quantization and Beyond](https://mlsanity.com/paper/2403.09673/) (2024)
- [LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookup](https://mlsanity.com/paper/2302.03213/) (2023)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings](https://mlsanity.com/paper/2204.00185/) (2022)
- [Neural Vector Fields: Implicit Representation by Explicit Learning](https://mlsanity.com/paper/2303.04341/) (2023)
- [FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clients](https://mlsanity.com/paper/2201.11865/) (2022)
- [Experimental Analysis of Large-scale Learnable Vector Storage Compression](https://mlsanity.com/paper/2311.15578/) (2023)
- [Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graph](https://mlsanity.com/paper/2208.05648/) (2022)
- [Representation Collapsing Problems in Vector Quantization](https://mlsanity.com/paper/2411.16550/) (2024)
- [Differentiable Product Quantization for Memory Efficient Camera Relocalization](https://mlsanity.com/paper/2407.15540/) (2024)
- [Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrieval](https://mlsanity.com/paper/2201.05409/) (2022)
- [Discrete Tokenization for Multimodal LLMs: A Comprehensive Survey](https://mlsanity.com/paper/2507.22920/) (2025)
- [PECAN: A Product-Quantized Content Addressable Memory Network](https://mlsanity.com/paper/2208.13571/) (2022)
- [Matching-oriented Product Quantization For Ad-hoc Retrieval](https://mlsanity.com/paper/2104.07858/) (2021)
- [i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systems](https://mlsanity.com/paper/2204.00281/) (2022)
- [MixQuant: Mixed Precision Quantization with a Bit-width Optimization Search](https://mlsanity.com/paper/2309.17341/) (2023)
- [Taxonomy of the Retrieval System Framework: Pitfalls and Paradigms](https://mlsanity.com/paper/2601.20131/) (2026)
- [MeToken: Uniform Micro-environment Token Boosts Post-Translational Modification Prediction](https://mlsanity.com/paper/2411.01856/) (2024)
- [Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes](https://mlsanity.com/paper/2203.05082/) (2022)
- [AdANNS: A Framework for Adaptive Semantic Search](https://mlsanity.com/paper/2305.19435/) (2023)
- [Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularization](https://mlsanity.com/paper/2309.01512/) (2023)
- [Set2Box: Similarity Preserving Representation Learning of Sets](https://mlsanity.com/paper/2210.03282/) (2022)
- [Stella Nera: A Differentiable Maddness-Based Hardware Accelerator for Efficient Approximate Matrix Multiplication](https://mlsanity.com/paper/2311.10207/) (2023)
- [Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenization](https://mlsanity.com/paper/2603.17052/) (2026)
- [FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compression](https://mlsanity.com/paper/2605.04084/) (2026)

## Abstract (arXiv)

Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of symbols and poses a critical challenge on memory and storage constraints. In this work, we propose a generic and end-to-end learnable compression framework termed differentiable product quantization (DPQ). We present two instantiations of DPQ that leverage different approximation techniques to enable differentiability in end-to-end learning. Our method can readily serve as a drop-in alternative for any existing embedding layer. Empirically, DPQ offers significant compression ratios (14-238×) at negligible or no performance cost on 10 datasets across three different language tasks.