# Retrieval-Oriented Knowledge for Click-Through Rate Prediction

arXiv: https://arxiv.org/abs/2404.18304 · HTML: https://mlsanity.com/paper/2404.18304/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Exploring Simple Siamese Representation Learning](https://mlsanity.com/paper/2011.10566/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [End-to-End User Behavior Retrieval in Click-Through RatePrediction Model](https://mlsanity.com/paper/2108.04468/) (2021)
- [Product-based Neural Networks for User Response Prediction over Multi-field Categorical Data](https://mlsanity.com/paper/1807.00311/) (2018)
- [User Behavior Retrieval for Click-Through Rate Prediction](https://mlsanity.com/paper/2005.14171/) (2020)
- [Lifelong Sequential Modeling with Personalized Memorization for User Response Prediction](https://mlsanity.com/paper/1905.00758/) (2019)
- [An Embedding Learning Framework for Numerical Features in CTR Prediction](https://mlsanity.com/paper/2012.08986/) (2020)
- [Feature Generation by Convolutional Neural Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1904.04447/) (2019)
- [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://mlsanity.com/paper/2205.10249/) (2022)
- [Retrieval & Interaction Machine for Tabular Data Prediction](https://mlsanity.com/paper/2108.05252/) (2021)
- [Towards Open-World Feature Extrapolation: An Inductive Graph Learning Approach](https://mlsanity.com/paper/2110.04514/) (2021)
- [An Adversarial Imitation Click Model for Information Retrieval](https://mlsanity.com/paper/2104.06077/) (2021)
- [IntTower: the Next Generation of Two-Tower Model for Pre-Ranking System](https://mlsanity.com/paper/2210.09890/) (2022)
- [A Graph-Enhanced Click Model for Web Search](https://mlsanity.com/paper/2206.08621/) (2022)
- [A Bird's-eye View of Reranking: from List Level to Page Level](https://mlsanity.com/paper/2211.09303/) (2022)
- [An F-shape Click Model for Information Retrieval on Multi-block Mobile Pages](https://mlsanity.com/paper/2206.08604/) (2022)
- [Learning Enhanced Representations for Tabular Data via Neighborhood Propagation](https://mlsanity.com/paper/2206.06587/) (2022)

## Built on by (most cited first, 1 of 1)

- [CtrlA: Adaptive Retrieval-Augmented Generation via Inherent Control](https://mlsanity.com/paper/2405.18727/) (2024)

## Abstract (arXiv)

Click-through rate (CTR) prediction is crucial for personalized online services. Sample-level retrieval-based models, such as RIM, have demonstrated remarkable performance. However, they face challenges including inference inefficiency and high resource consumption due to the retrieval process, which hinder their practical application in industrial settings. To address this, we propose a universal plug-and-play underlineretrieval-underlineoriented underlineknowledge (name) framework that bypasses the real retrieval process. The framework features a knowledge base that preserves and imitates the retrieved & aggregated representations using a decomposition-reconstruction paradigm. Knowledge distillation and contrastive learning optimize the knowledge base, enabling the integration of retrieval-enhanced representations with various CTR models. Experiments on three large-scale datasets demonstrate name's exceptional compatibility and performance, with the neural knowledge base serving as an effective surrogate for the retrieval pool. name surpasses the teacher model while maintaining superior inference efficiency and demonstrates the feasibility of distilling knowledge from non-parametric methods using a parametric approach. These results highlight name's strong potential for real-world applications and its ability to transform retrieval-based methods into practical solutions. Our implementation code is available to support reproducibility in urlhttps://github.com/HSLiu-Initial/ROK.git.