# Towards a Theoretical Understanding of Two Tower Recommendation Models

arXiv: https://arxiv.org/abs/2403.00802 · HTML: https://mlsanity.com/paper/2403.00802/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 62 references that are in the map.
- First posted on arXiv 23 Feb 2024.

## Builds on (most cited first, 19 of 19)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [DCN V2: Improved Deep & Cross Network and Practical Lessons for Web-scale Learning to Rank Systems](https://mlsanity.com/paper/2008.13535/) (2020)
- [Top-K Off-Policy Correction for a REINFORCE Recommender System](https://mlsanity.com/paper/1812.02353/) (2018)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [Pixie: A System for Recommending 3+ Billion Items to 200+ Million Users in Real-Time](https://mlsanity.com/paper/1711.07601/) (2017)
- [Graph Enhanced Representation Learning for News Recommendation](https://mlsanity.com/paper/2003.14292/) (2020)
- [Bootstrapping User and Item Representations for One-Class Collaborative Filtering](https://mlsanity.com/paper/2105.06323/) (2021)
- [Fairly Adaptive Negative Sampling for Recommendations](https://mlsanity.com/paper/2302.08266/) (2023)
- [Candidate Generation with Binary Codes for Large-Scale Top-N Recommendation](https://mlsanity.com/paper/1909.05475/) (2019)
- [Deconfounded Causal Collaborative Filtering](https://mlsanity.com/paper/2110.07122/) (2021)
- [MetaSelector: Meta-Learning for Recommendation with User-Level Adaptive Model Selection](https://mlsanity.com/paper/2001.10378/) (2020)
- [Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation](https://mlsanity.com/paper/2311.18213/) (2023)
- [Robust Collaborative Filtering to Popularity Distribution Shift](https://mlsanity.com/paper/2310.10696/) (2023)
- [Smooth Bandit Optimization: Generalization to H"older Space](https://mlsanity.com/paper/2012.06076/) (2020)
- [The Minority Matters: A Diversity-Promoting Collaborative Metric Learning Algorithm](https://mlsanity.com/paper/2209.15292/) (2022)
- [Deep Unified Representation for Heterogeneous Recommendation](https://mlsanity.com/paper/2201.05861/) (2022)
- [Dynamic Causal Collaborative Filtering](https://mlsanity.com/paper/2208.11094/) (2022)

## Abstract (arXiv)

Production-grade recommender systems rely heavily on a large-scale corpus used by online media services, including Netflix, Pinterest, and Amazon. These systems enrich recommendations by learning users' and items' embeddings projected in a low-dimensional space with two tower models (two deep neural networks), which facilitate their embedding constructs to predict users' feedback associated with items. Despite its popularity for recommendations, its theoretical behaviors remain comprehensively unexplored. We study the asymptotic behaviors of the two tower model applied in two-stage recommenders that entail a strong convergence to the optimal recommender system. We establish certain theoretical properties and statistical assurance of the two tower recommender. In addition to asymptotic behaviors, we demonstrate that recommendation with two tower architecture attains faster convergence by relying on the intrinsic dimensions of the input features. Finally, we show numerically that the two tower recommender enables encapsulating the impacts of items' and users' attributes on ratings, resulting in better performance compared to existing methods conducted using synthetic and real-world data experiments.