# RAT: Retrieval-Augmented Transformer for Click-Through Rate Prediction

arXiv: https://arxiv.org/abs/2404.02249 · HTML: https://mlsanity.com/paper/2404.02249/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Generation and context integration, one of 15 directions in Retrieval-augmented generation.
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Placed in

- Retrieval-augmented generation / [Generation and context integration](https://mlsanity.com/rag/generation-and-context-integration/)

## Builds on (most cited first, 5 of 5)

- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09433/) (2019)
- [BARS: Towards Open Benchmarking for Recommender Systems](https://mlsanity.com/paper/2205.09626/) (2022)
- [MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation](https://mlsanity.com/paper/2308.11175/) (2023)

## Built on by (most cited first, 1 of 1)

- [Retrieval-Oriented Knowledge for Click-Through Rate Prediction](https://mlsanity.com/paper/2404.18304/) (2024)

## Abstract (arXiv)

Predicting click-through rates (CTR) is a fundamental task for Web applications, where a key issue is to devise effective models for feature interactions. Current methodologies predominantly concentrate on modeling feature interactions within an individual sample, while overlooking the potential cross-sample relationships that can serve as a reference context to enhance the prediction. To make up for such deficiency, this paper develops a Retrieval-Augmented Transformer (RAT), aiming to acquire fine-grained feature interactions within and across samples. By retrieving similar samples, we construct augmented input for each target sample. We then build Transformer layers with cascaded attention to capture both intra- and cross-sample feature interactions, facilitating comprehensive reasoning for improved CTR prediction while retaining efficiency. Extensive experiments on real-world datasets substantiate the effectiveness of RAT and suggest its advantage in long-tail scenarios. The code has been open-sourced at urlhttps://github.com/YushenLi807/WWW24-RAT.