# Revisiting Neural Retrieval on Accelerators

arXiv: https://arxiv.org/abs/2306.04039 · HTML: https://mlsanity.com/paper/2306.04039/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 58 references that are in the map.
- First posted on arXiv 6 Jun 2023.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Mixed Precision Training](https://mlsanity.com/paper/1710.03740/) (2017)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep & Cross Network for Ad Click Predictions](https://mlsanity.com/paper/1708.05123/) (2017)
- [FP8 Formats for Deep Learning](https://mlsanity.com/paper/2209.05433/) (2022)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Breaking the Softmax Bottleneck: A High-Rank RNN Language Model](https://mlsanity.com/paper/1711.03953/) (2017)
- [Product-based Neural Networks for User Response Prediction](https://mlsanity.com/paper/1611.00144/) (2016)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [Neural Collaborative Filtering vs. Matrix Factorization Revisited](https://mlsanity.com/paper/2005.09683/) (2020)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Asymmetric LSH (ALSH) for Sublinear Time Maximum Inner Product Search (MIPS)](https://mlsanity.com/paper/1405.5869/) (2014)
- [Billion-scale Commodity Embedding for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1803.02349/) (2018)
- [Learning Tree-based Deep Model for Recommender Systems](https://mlsanity.com/paper/1801.02294/) (2018)
- [Self-Supervised Reinforcement Learning for Recommender Systems](https://mlsanity.com/paper/2006.05779/) (2020)
- [Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time](https://mlsanity.com/paper/1711.07601/) (2017)
- [FBGEMM: Enabling High-Performance Low-Precision Deep Learning Inference](https://mlsanity.com/paper/2101.05615/) (2021)
- [Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems](https://mlsanity.com/paper/2005.12964/) (2020)
- [Sparse-Interest Network for Sequential Recommendation](https://mlsanity.com/paper/2102.09267/) (2021)
- [Joint Optimization of Tree-based Index and Deep Model for Recommender Systems](https://mlsanity.com/paper/1902.07565/) (2019)

## Built on by (most cited first, 14 of 14)

- [Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations](https://mlsanity.com/paper/2402.17152/) (2024)
- [Temporal Interest Network for User Response Prediction](https://mlsanity.com/paper/2308.08487/) (2023)
- [Preference Diffusion for Recommendation](https://mlsanity.com/paper/2410.13117/) (2024)
- [DV365: Extremely Long User History Modeling at Instagram](https://mlsanity.com/paper/2506.00450/) (2025)
- [Long-Sequence Recommendation Models Need Decoupled Embeddings](https://mlsanity.com/paper/2410.02604/) (2024)
- [Request-Only Optimization for Recommendation Systems](https://mlsanity.com/paper/2508.05640/) (2025)
- [Large Language Model as Universal Retriever in Industrial-Scale Recommender System](https://mlsanity.com/paper/2502.03041/) (2025)
- [Retrieval with Learned Similarities](https://mlsanity.com/paper/2407.15462/) (2024)
- [LiNR: Model Based Neural Retrieval on GPUs at LinkedIn](https://mlsanity.com/paper/2407.13218/) (2024)
- [Finding Interest Needle in Popularity Haystack: Improving Retrieval by Modeling Item Exposure](https://mlsanity.com/paper/2503.23630/) (2025)
- [RADAR: Recall Augmentation through Deferred Asynchronous Retrieval](https://mlsanity.com/paper/2506.07261/) (2025)
- [Closing the Performance Gap in Generative Recommenders with Collaborative Tokenization and Efficient Modeling](https://mlsanity.com/paper/2508.14910/) (2025)
- [Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval](https://mlsanity.com/paper/2607.00448/) (2026)
- [OneShot: Index-in-Ranking with Neural Scoring for Large-Scale Retrieval](https://mlsanity.com/paper/2607.27475/) (2026)

## Abstract (arXiv)

Retrieval finds a small number of relevant candidates from a large corpus for information retrieval and recommendation applications. A key component of retrieval is to model (user, item) similarity, which is commonly represented as the dot product of two learned embeddings. This formulation permits efficient inference, commonly known as Maximum Inner Product Search (MIPS). Despite its popularity, dot products cannot capture complex user-item interactions, which are multifaceted and likely high rank. We hence examine non-dot-product retrieval settings on accelerators, and propose textitmixture of logits (MoL), which models (user, item) similarity as an adaptive composition of elementary similarity functions. This new formulation is expressive, capable of modeling high rank (user, item) interactions, and further generalizes to the long tail. When combined with a hierarchical retrieval strategy, textith-indexer, we are able to scale up MoL to 100M corpus on a single GPU with latency comparable to MIPS baselines. On public datasets, our approach leads to uplifts of up to 77.3% in hit rate (HR). Experiments on a large recommendation surface at Meta showed strong metric gains and reduced popularity bias, validating the proposed approach's performance and improved generalization.