# Ads Recommendation in a Collapsed and Entangled World

arXiv: https://arxiv.org/abs/2403.00793 · HTML: https://mlsanity.com/paper/2403.00793/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 49 of its 79 references that are in the map.
- First posted on arXiv 22 Feb 2024.

## Builds on (most cited first, 30 of 49)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Understanding Dimensional Collapse in Contrastive Self-supervised Learning](https://mlsanity.com/paper/2110.09348/) (2021)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [AutoInt: Automatic Feature Interaction Learning via Self-Attentive Neural Networks](https://mlsanity.com/paper/1810.11921/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [xDeepFM: Combining Explicit and Implicit Feature Interactions for Recommender Systems](https://mlsanity.com/paper/1803.05170/) (2018)
- [MultiModal-GPT: A Vision and Language Model for Dialogue with Humans](https://mlsanity.com/paper/2305.04790/) (2023)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [Adapting Large Language Models by Integrating Collaborative Semantics for Recommendation](https://mlsanity.com/paper/2311.09049/) (2023)
- [Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Prediction](https://mlsanity.com/paper/2006.05639/) (2020)
- [On Feature Decorrelation in Self-Supervised Learning](https://mlsanity.com/paper/2105.00470/) (2021)
- [Product-based Neural Networks for User Response Prediction](https://mlsanity.com/paper/1611.00144/) (2016)
- [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://mlsanity.com/paper/1905.06874/) (2019)
- [Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisited](https://mlsanity.com/paper/2303.13835/) (2023)
- [Towards Universal Sequence Representation Learning for Recommender Systems](https://mlsanity.com/paper/2206.05941/) (2022)
- [Learning Vector-Quantized Item Representation for Transferable Sequential Recommenders](https://mlsanity.com/paper/2210.12316/) (2022)
- [Learning Disentangled Representations for Recommendation](https://mlsanity.com/paper/1910.14238/) (2019)
- [FiBiNET: Combining Feature Importance and Bilinear feature Interaction for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09433/) (2019)

## Built on by (most cited first, 13 of 13)

- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [External Large Foundation Model: How to Efficiently Serve Trillions of Parameters for Online Ads Recommendation](https://mlsanity.com/paper/2502.17494/) (2025)
- [Pre-train, Align, and Disentangle: Empowering Sequential Recommendation with Large Language Models](https://mlsanity.com/paper/2412.04107/) (2024)
- [Towards Scalable Semantic Representation for Recommendation](https://mlsanity.com/paper/2410.09560/) (2024)
- [Revisiting Feature Interactions from the Perspective of Quadratic Neural Networks for Click-through Rate Prediction](https://mlsanity.com/paper/2505.17999/) (2025)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [OmniReview: A Large-scale Benchmark and LLM-enhanced Framework for Realistic Reviewer Recommendation](https://mlsanity.com/paper/2602.08896/) (2026)
- [Expand More, Shrink Less: Shaping Effective-Rank Dynamics for Dense Scaling in Recommendation](https://mlsanity.com/paper/2605.23191/) (2026)
- [Taming Recommendation Bias with Causal Intervention on Evolving Personal Popularity](https://mlsanity.com/paper/2505.14310/) (2025)
- [Distribution-Aware End-to-End Embedding for Streaming Numerical Features in Click-Through Rate Prediction](https://mlsanity.com/paper/2602.03223/) (2026)
- [Understanding DNNs in Feature Interaction Models: A Dimensional Collapse Perspective](https://mlsanity.com/paper/2604.26489/) (2026)
- [Interest Entanglement: The Hidden Barrier to Blind Super-Resolution Optimization](https://mlsanity.com/paper/2606.22353/) (2026)
- [MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search](https://mlsanity.com/paper/2607.11030/) (2026)

## Abstract (arXiv)

We present Tencent's ads recommendation system and examine the challenges and practices of learning appropriate recommendation representations. Our study begins by showcasing our approaches to preserving prior knowledge when encoding features of diverse types into embedding representations. We specifically address sequence features, numeric features, and pre-trained embedding features. Subsequently, we delve into two crucial challenges related to feature representation: the dimensional collapse of embeddings and the interest entanglement across different tasks or scenarios. We propose several practical approaches to address these challenges that result in robust and disentangled recommendation representations. We then explore several training techniques to facilitate model optimization, reduce bias, and enhance exploration. Additionally, we introduce three analysis tools that enable us to study feature correlation, dimensional collapse, and interest entanglement. This work builds upon the continuous efforts of Tencent's ads recommendation team over the past decade. It summarizes general design principles and presents a series of readily applicable solutions and analysis tools. The reported performance is based on our online advertising platform, which handles hundreds of billions of requests daily and serves millions of ads to billions of users.