# Ad-Rec: Advanced Feature Interactions to Address Covariate-Shifts in Recommendation Networks

arXiv: https://arxiv.org/abs/2308.14902 · HTML: https://mlsanity.com/paper/2308.14902/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 41 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 19 of 19)

- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Attentional Factorization Machines: Learning the Weight of Feature Interactions via Attention Networks](https://mlsanity.com/paper/1708.04617/) (2017)
- [The Architectural Implications of Facebook's DNN-based Personalized Recommendation](https://mlsanity.com/paper/1906.03109/) (2019)
- [Fi-GNN: Modeling Feature Interactions via Graph Neural Networks for CTR Prediction](https://mlsanity.com/paper/1910.05552/) (2019)
- [AutoFIS: Automatic Feature Interaction Selection in Factorization Models for Click-Through Rate Prediction](https://mlsanity.com/paper/2003.11235/) (2020)
- [Understanding Scaling Laws for Recommendation Models](https://mlsanity.com/paper/2208.08489/) (2022)
- [Detecting Beneficial Feature Interactions for Recommender Systems](https://mlsanity.com/paper/2008.00404/) (2020)
- [Accelerating Recommendation System Training by Leveraging Popular Choices](https://mlsanity.com/paper/2103.00686/) (2021)
- [Heterogeneous Acceleration Pipeline for Recommendation System Training](https://mlsanity.com/paper/2204.05436/) (2022)
- [Time-based Sequence Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/2008.11922/) (2020)
- [AIM: Automatic Interaction Machine for Click-Through Rate Prediction](https://mlsanity.com/paper/2111.03318/) (2021)

## Built on by (most cited first, 3 of 3)

- [Keyformer: KV Cache Reduction through Key Tokens Selection for Efficient Generative Inference](https://mlsanity.com/paper/2403.09054/) (2024)
- [Disaggregated Multi-Tower: Topology-aware Modeling Technique for Efficient Large-Scale Recommendation](https://mlsanity.com/paper/2403.00877/) (2024)
- [VectorLiteRAG: Latency-Aware and Fine-Grained Resource Partitioning for Efficient RAG](https://mlsanity.com/paper/2504.08930/) (2025)

## Abstract (arXiv)

Recommendation models are vital in delivering personalized user experiences by leveraging the correlation between multiple input features. However, deep learning-based recommendation models often face challenges due to evolving user behaviour and item features, leading to covariate shifts. Effective cross-feature learning is crucial to handle data distribution drift and adapting to changing user behaviour. Traditional feature interaction techniques have limitations in achieving optimal performance in this context. This work introduces Ad-Rec, an advanced network that leverages feature interaction techniques to address covariate shifts. This helps eliminate irrelevant interactions in recommendation tasks. Ad-Rec leverages masked transformers to enable the learning of higher-order cross-features while mitigating the impact of data distribution drift. Our approach improves model quality, accelerates convergence, and reduces training time, as measured by the Area Under Curve (AUC) metric. We demonstrate the scalability of Ad-Rec and its ability to achieve superior model quality through comprehensive ablation studies.