mlsanityData to 7 Oct 2026

Differentiable Product Quantization for End-to-End Embedding Compression

arXiv 1908.09756 ↗First posted 26 Aug 2019cs.LG cs.AI cs.CLcited by 36

Not on the map. Why?

Abstract (arXiv)

Embedding layers are commonly used to map discrete symbols into continuous embedding vectors that reflect their semantic meanings. Despite their effectiveness, the number of parameters in an embedding layer increases linearly with the number of symbols and poses a critical challenge on memory and storage constraints. In this work, we propose a generic and end-to-end learnable compression framework termed differentiable product quantization (DPQ). We present two instantiations of DPQ that leverage different approximation techniques to enable differentiability in end-to-end learning. Our method can readily serve as a drop-in alternative for any existing embedding layer. Empirically, DPQ offers significant compression ratios (14-238×) at negligible or no performance cost on 10 datasets across three different language tasks.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 26 Aug 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 10; 2023: 11; 2024: 6; 2025: 3; 2026: 4.

36 papers.

Sort:

By area:

  1. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  2. 2022Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioningcited by 362
  3. 2022Learning Vector-Quantized Item Representation for Transferable Sequential Recommenderscited by 112
  4. 2024VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Modelscited by 53
  5. 2024FoldToken: Learning Protein Language via Vector Quantization and Beyondcited by 21
  6. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 19
  7. 2023Embedding in Recommender Systems: A Surveycited by 19
  8. 2023IRGen: Generative Modeling for Image Retrievalcited by 18
  9. 2022Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddingscited by 17
  10. 2023Neural Vector Fields: Implicit Representation by Explicit Learningcited by 16
  11. 2022FedLite: A Scalable Approach for Federated Learning on Resource-constrained Clientscited by 14
  12. 2023Experimental Analysis of Large-scale Learnable Vector Storage Compressioncited by 12
24 more
  1. 2022Embedding Compression with Hashing for Efficient Representation Learning in Large-Scale Graphcited by 10
  2. 2024Representation Collapsing Problems in Vector Quantizationcited by 10
  3. 2024Differentiable Product Quantization for Memory Efficient Camera Relocalizationcited by 9
  4. 2022Progressively Optimized Bi-Granular Document Representation for Scalable Embedding Based Retrievalcited by 7
  5. 2025Discrete Tokenization for Multimodal LLMs: A Comprehensive Surveycited by 7
  6. 2022PECAN: A Product-Quantized Content Addressable Memory Networkcited by 6
  7. 2021Matching-oriented Product Quantization For Ad-hoc Retrievalcited by 5
  8. 2022i-Razor: A Differentiable Neural Input Razor for Feature Selection and Dimension Search in DNN-Based Recommender Systemscited by 4
  9. 2023MixQuant: Mixed Precision Quantization with a Bit-width Optimization Searchcited by 4
  10. 2026Taxonomy of the Retrieval System Framework: Pitfalls and Paradigmscited by 4
  11. 2024MeToken: Uniform Micro-environment Token Boosts Post-Translational Modification Predictioncited by 3
  12. 2022Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexescited by 2
  13. 2023AdANNS: A Framework for Adaptive Semantic Searchcited by 2
  14. 2023Neural Vector Fields: Generalizing Distance Vector Fields by Codebooks and Zero-Curl Regularizationcited by 2
  15. 2022Set2Box: Similarity Preserving Representation Learning of Setscited by 1
  16. 2023Stella Nera: A Differentiable Maddness-Based Hardware Accelerator for Efficient Approximate Matrix Multiplicationcited by 1
  17. 2026Early Quantization Shrinks Codebook: A Simple Fix for Diversity-Preserving Tokenizationcited by 1
  18. 2026FASQ: Flexible Accelerated Subspace Quantization for Calibration-Free LLM Compressioncited by 1
  19. 2021Unsupervised Domain-adaptive Hash for Networkscited by 0
  20. 2023Training A Multi-stage Deep Classifier with Feedback Signalscited by 0
  21. 2024Distilling Vision-Language Pretraining for Efficient Cross-Modal Retrievalcited by 0
  22. 2025Natural Language Satisfiability: Exploring the Problem Distribution and Evaluating Transformer-based Language Modelscited by 0
  23. 2025Semantic Compression via Multimodal Representation Learningcited by 0
  24. 2026OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrievalcited by 0