# RECE: Reduced Cross-Entropy Loss for Large-Catalogue Sequential Recommenders

arXiv: https://arxiv.org/abs/2408.02354 · HTML: https://mlsanity.com/paper/2408.02354/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 42 references that are in the map.
- First posted on arXiv 5 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Reformer: The Efficient Transformer](https://mlsanity.com/paper/2001.04451/) (2020)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Contrastive Learning for Representation Degeneration Problem in Sequential Recommendation](https://mlsanity.com/paper/2110.05730/) (2021)
- [Practical and Optimal LSH for Angular Distance](https://mlsanity.com/paper/1509.02897/) (2015)
- [Frequency Enhanced Hybrid Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2304.09184/) (2023)
- [Exploring Data Splitting Strategies for the Evaluation of Recommendation Models](https://mlsanity.com/paper/2007.13237/) (2020)
- [A Critical Study on Data Leakage in Recommender System Offline Evaluation](https://mlsanity.com/paper/2010.11060/) (2020)
- [Simplify and Robustify Negative Sampling for Implicit Collaborative Filtering](https://mlsanity.com/paper/2009.03376/) (2020)
- [Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?](https://mlsanity.com/paper/2309.07602/) (2023)
- [Quantization based Fast Inner Product Search](https://mlsanity.com/paper/1509.01469/) (2015)
- [gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling](https://mlsanity.com/paper/2308.07192/) (2023)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [Cross-Batch Negative Sampling for Training Two-Tower Recommenders](https://mlsanity.com/paper/2110.15154/) (2021)
- [Vista: A Visually, Socially, and Temporally-aware Model for Artistic Recommendation](https://mlsanity.com/paper/1607.04373/) (2016)
- [A New Unbiased and Efficient Class of LSH-Based Samplers and Estimators for Partition Function Computation in Log-Linear Models](https://mlsanity.com/paper/1703.05160/) (2017)
- [Contrastive Learning with Bidirectional Transformers for Sequential Recommendation](https://mlsanity.com/paper/2208.03895/) (2022)
- [Scaling Session-Based Transformer Recommendations using Optimized Negative Sampling and Loss Functions](https://mlsanity.com/paper/2307.14906/) (2023)
- [Deep Networks With Large Output Spaces](https://mlsanity.com/paper/1412.7479/) (2014)
- [Tensor-based Sequential Learning via Hankel Matrix Representation for Next Item Recommendations](https://mlsanity.com/paper/2212.05720/) (2022)
- [Generating Negative Samples for Sequential Recommendation](https://mlsanity.com/paper/2208.03645/) (2022)

## Built on by (most cited first, 9 of 9)

- [Scalable Cross-Entropy Loss for Sequential Recommendations with Large Item Catalogs](https://mlsanity.com/paper/2409.18721/) (2024)
- [Time to Split: Exploring Data Splitting Strategies for Offline Evaluation of Sequential Recommenders](https://mlsanity.com/paper/2507.16289/) (2025)
- [Autoregressive Generation Strategies for Top-K Sequential Recommendations](https://mlsanity.com/paper/2409.17730/) (2024)
- [Recommendation Is a Dish Better Served Warm](https://mlsanity.com/paper/2508.07856/) (2025)
- [Maximum Impact with Fewer Features: Efficient Feature Selection for Cold-Start Recommenders through Collaborative Importance Weighting](https://mlsanity.com/paper/2508.06455/) (2025)
- [Position-Aware Sequential Attention for Accurate Next Item Recommendations](https://mlsanity.com/paper/2602.21052/) (2026)
- [Cross-Representation Knowledge Transfer for Improved Sequential Recommendations](https://mlsanity.com/paper/2602.23471/) (2026)
- [Batch Size or Negatives? A Selection Rule for Memory-Constrained Recommender Training](https://mlsanity.com/paper/2608.11061/) (2026)
- [Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation](https://mlsanity.com/paper/2608.24079/) (2026)

## Abstract (arXiv)

Scalability is a major challenge in modern recommender systems. In sequential recommendations, full Cross-Entropy (CE) loss achieves state-of-the-art recommendation quality but consumes excessive GPU memory with large item catalogs, limiting its practicality. Using a GPU-efficient locality-sensitive hashing-like algorithm for approximating large tensor of logits, this paper introduces a novel RECE (REduced Cross-Entropy) loss. RECE significantly reduces memory consumption while allowing one to enjoy the state-of-the-art performance of full CE loss. Experimental results on various datasets show that RECE cuts training peak memory usage by up to 12 times compared to existing methods while retaining or exceeding performance metrics of CE loss. The approach also opens up new possibilities for large-scale applications in other domains.