# Unleashing the Potential of Two-Tower Models: Diffusion-Based Cross-Interaction for Large-Scale Matching

arXiv: https://arxiv.org/abs/2502.20687 · HTML: https://mlsanity.com/paper/2502.20687/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 39 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://mlsanity.com/paper/2004.12832/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [Deep Interest Evolution Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1809.03672/) (2018)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [ProDiff: Progressive Fast Diffusion Model For High-Quality Text-to-Speech](https://mlsanity.com/paper/2207.06389/) (2022)
- [DiffuRec: A Diffusion Model for Sequential Recommendation](https://mlsanity.com/paper/2304.00686/) (2023)
- [Diffusion Recommender Model](https://mlsanity.com/paper/2304.04971/) (2023)
- [KuaiRand: An Unbiased Sequential Recommendation Dataset with Randomly Exposed Videos](https://mlsanity.com/paper/2208.08696/) (2022)
- [SimpleX: A Simple and Strong Baseline for Collaborative Filtering](https://mlsanity.com/paper/2109.12613/) (2021)
- [Deep Session Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.06482/) (2019)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [COLD: Towards the Next Generation of Pre-Ranking System](https://mlsanity.com/paper/2007.16122/) (2020)
- [Graph Signal Diffusion Model for Collaborative Filtering](https://mlsanity.com/paper/2311.08744/) (2023)
- [ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendation](https://mlsanity.com/paper/2209.00456/) (2022)
- [IntTower: the Next Generation of Two-Tower Model for Pre-Ranking System](https://mlsanity.com/paper/2210.09890/) (2022)
- [Collaborative Filtering Based on Diffusion Models: Unveiling the Potential of High-Order Connectivity](https://mlsanity.com/paper/2404.14240/) (2024)
- [Towards a Better Tradeoff between Effectiveness and Efficiency in Pre-Ranking: A Learnable Feature Selection based Approach](https://mlsanity.com/paper/2105.07706/) (2021)
- [Beyond Two-Tower Matching: Learning Sparse Retrievable Cross-Interactions for Recommendation](https://mlsanity.com/paper/2311.18213/) (2023)
- [Accelerating Large-Scale Inference with Anisotropic Vector Quantization](https://mlsanity.com/paper/1908.10396/) (2019)
- [Divide and Conquer: Towards Better Embedding-based Retrieval for Recommender Systems From a Multi-task Perspective](https://mlsanity.com/paper/2302.02657/) (2023)
- [Exploration in two-stage recommender systems](https://mlsanity.com/paper/2009.08956/) (2020)

## Built on by (most cited first, 3 of 3)

- [Extracting Interaction-Aware Monosemantic Concepts in Recommender Systems](https://mlsanity.com/paper/2511.18024/) (2025)
- [Eval4DiRec: A Unified and Systematic Evaluation Framework for Diffusion-based Recommender Systems](https://mlsanity.com/paper/2609.34404/) (2026)
- [A Simple and Effective Framework for Symmetric Consistent Indexing in Large-Scale Dense Retrieval](https://mlsanity.com/paper/2512.13074/) (2025)

## Abstract (arXiv)

Two-tower models are widely adopted in the industrial-scale matching stage across a broad range of application domains, such as content recommendations, advertisement systems, and search engines. This model efficiently handles large-scale candidate item screening by separating user and item representations. However, the decoupling network also leads to a neglect of potential information interaction between the user and item representations. Current state-of-the-art (SOTA) approaches include adding a shallow fully connected layer(i.e., COLD), which is limited by performance and can only be used in the ranking stage. For performance considerations, another approach attempts to capture historical positive interaction information from the other tower by regarding them as the input features(i.e., DAT). Later research showed that the gains achieved by this method are still limited because of lacking the guidance on the next user intent. To address the aforementioned challenges, we propose a "cross-interaction decoupling architecture" within our matching paradigm. This user-tower architecture leverages a diffusion module to reconstruct the next positive intention representation and employs a mixed-attention module to facilitate comprehensive cross-interaction. During the next positive intention generation, we further enhance the accuracy of its reconstruction by explicitly extracting the temporal drift within user behavior sequences. Experiments on two real-world datasets and one industrial dataset demonstrate that our method outperforms the SOTA two-tower models significantly, and our diffusion approach outperforms other generative models in reconstructing item representations.