# Joint Learning of Deep Retrieval Model and Product Quantization based Embedding Index

arXiv: https://arxiv.org/abs/2105.03933 · HTML: https://mlsanity.com/paper/2105.03933/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2021.

## Built on by (most cited first, 11 of 11)

- [IRGen: Generative Modeling for Image Retrieval](https://mlsanity.com/paper/2303.10126/) (2023)
- [Distill-VQ: Learning Retrieval Oriented Vector Quantization By Distilling Knowledge from Dense Embeddings](https://mlsanity.com/paper/2204.00185/) (2022)
- [Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation](https://mlsanity.com/paper/2311.18213/) (2023)
- [Generative Retrieval with Preference Optimization for E-commerce Search](https://mlsanity.com/paper/2407.19829/) (2024)
- [Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval](https://mlsanity.com/paper/2202.03384/) (2022)
- [Adaptive In-Context Learning with Large Language Models for Bundle Generation](https://mlsanity.com/paper/2312.16262/) (2023)
- [Lexically-Accelerated Dense Retrieval](https://mlsanity.com/paper/2307.16779/) (2023)
- [Attention Weighted Mixture of Experts with Contrastive Learning for Personalized Ranking in E-commerce](https://mlsanity.com/paper/2306.05011/) (2023)
- [Differentiable Retrieval Augmentation via Generative Language Modeling for E-commerce Query Intent Classification](https://mlsanity.com/paper/2308.09308/) (2023)
- [Givens Coordinate Descent Methods for Rotation Matrix Learning in Trainable Embedding Indexes](https://mlsanity.com/paper/2203.05082/) (2022)
- [pEBR: A Probabilistic Approach to Embedding Based Retrieval](https://mlsanity.com/paper/2410.19349/) (2024)

## Abstract (arXiv)

Embedding index that enables fast approximate nearest neighbor(ANN) search, serves as an indispensable component for state-of-the-art deep retrieval systems. Traditional approaches, often separating the two steps of embedding learning and index building, incur additional indexing time and decayed retrieval accuracy. In this paper, we propose a novel method called Poeem, which stands for product quantization based embedding index jointly trained with deep retrieval model, to unify the two separate steps within an end-to-end training, by utilizing a few techniques including the gradient straight-through estimator, warm start strategy, optimal space decomposition and Givens rotation. Extensive experimental results show that the proposed method not only improves retrieval accuracy significantly but also reduces the indexing time to almost none. We have open sourced our approach for the sake of comparison and reproducibility.