# Cross-Batch Negative Sampling for Training Two-Tower Recommenders

arXiv: https://arxiv.org/abs/2110.15154 · HTML: https://mlsanity.com/paper/2110.15154/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Oct 2021.

## Built on by (most cited first, 24 of 24)

- [Your Group-Relative Advantage Is Biased](https://mlsanity.com/paper/2601.08521/) (2026)
- [BRIGHT -- Graph Neural Networks in Real-Time Fraud Detection](https://mlsanity.com/paper/2205.13084/) (2022)
- [Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs](https://mlsanity.com/paper/2409.18721/) (2024)
- [Revisiting Neural Retrieval on Accelerators](https://mlsanity.com/paper/2306.04039/) (2023)
- [Que2Engage: Embedding-based Retrieval for Relevant and Engaging Products at Facebook Marketplace](https://mlsanity.com/paper/2302.11052/) (2023)
- [RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders](https://mlsanity.com/paper/2408.02354/) (2024)
- [HyperFormer: Learning Expressive Sparse Feature Representations via Hypergraph Transformer](https://mlsanity.com/paper/2305.17386/) (2023)
- [FedCL: Federated Contrastive Learning for Privacy-Preserving Recommendation](https://mlsanity.com/paper/2204.09850/) (2022)
- [X-Reflect: Cross-Reflection Prompting for Multimodal Recommendation](https://mlsanity.com/paper/2408.15172/) (2024)
- [EAGER-LLM: Enhancing Large Language Models as Recommenders through Exogenous Behavior-Semantic Integration](https://mlsanity.com/paper/2502.14735/) (2025)
- [Global Contrastive Batch Sampling via Optimization on Sample Permutations](https://mlsanity.com/paper/2210.12874/) (2022)
- [Neighborhood-based Hard Negative Mining for Sequential Recommendation](https://mlsanity.com/paper/2306.10047/) (2023)
- [Evaluating Performance and Bias of Negative Sampling in Large-Scale Sequential Recommendation Models](https://mlsanity.com/paper/2410.17276/) (2024)
- [A Hybrid Cross-Stage Coordination Pre-ranking Model for Online Recommendation Systems](https://mlsanity.com/paper/2502.10284/) (2025)
- [GFlowGR: Fine-tuning Generative Recommendation Frameworks with Generative Flow Networks](https://mlsanity.com/paper/2506.16114/) (2025)
- [Delving into E-Commerce Product Retrieval with Vision-Language Pre-training](https://mlsanity.com/paper/2304.04377/) (2023)
- [Handling Large-scale Cardinality in building recommendation systems](https://mlsanity.com/paper/2401.09572/) (2024)
- [Policy Improvement Reinforcement Learning](https://mlsanity.com/paper/2604.00860/) (2026)
- [Ranking Differential Privacy](https://mlsanity.com/paper/2301.00841/) (2023)
- [SciTopic: Enhancing Topic Discovery in Scientific Literature through Advanced LLM](https://mlsanity.com/paper/2508.20514/) (2025)
- [DPR: An Algorithm Mitigate Bias Accumulation in Recommendation feedback loops](https://mlsanity.com/paper/2311.05864/) (2023)
- [Real-Time Hard Negative Sampling via LLM-based Clustering for Large-Scale Two-Tower Retrieval](https://mlsanity.com/paper/2607.00448/) (2026)
- [Don't Contrast the Impossible: Region-Constrained Batching for Contrastive User Modeling on a Local Community Platform](https://mlsanity.com/paper/2607.28971/) (2026)
- [Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation](https://mlsanity.com/paper/2608.24079/) (2026)

## Abstract (arXiv)

The two-tower architecture has been widely applied for learning item and user representations, which is important for large-scale recommender systems. Many two-tower models are trained using various in-batch negative sampling strategies, where the effects of such strategies inherently rely on the size of mini-batches. However, training two-tower models with a large batch size is inefficient, as it demands a large volume of memory for item and user contents and consumes a lot of time for feature encoding. Interestingly, we find that neural encoders can output relatively stable features for the same input after warming up in the training process. Based on such facts, we propose a simple yet effective sampling strategy called Cross-Batch Negative Sampling (CBNS), which takes advantage of the encoded item embeddings from recent mini-batches to boost the model training. Both theoretical analysis and empirical evaluations demonstrate the effectiveness and the efficiency of CBNS.