# Taming the One-Epoch Phenomenon in Online Recommendation System by Two-stage Contrastive ID Pre-training

arXiv: https://arxiv.org/abs/2508.18700 · HTML: https://mlsanity.com/paper/2508.18700/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 15 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Builds on (most cited first, 9 of 9)

- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [Practice on Long Sequential User Behavior Modeling for Click-Through Rate Prediction](https://mlsanity.com/paper/1905.09248/) (2019)
- [TT-Rec: Tensor Train Compression for Deep Learning Recommendation Models](https://mlsanity.com/paper/2101.11714/) (2021)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [Open Benchmarking for Click-Through Rate Prediction](https://mlsanity.com/paper/2009.05794/) (2020)
- [Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations](https://mlsanity.com/paper/2306.08121/) (2023)
- [On the Factory Floor: ML Engineering for Industrial-Scale Ads Recommendation Models](https://mlsanity.com/paper/2209.05310/) (2022)
- [Towards Understanding the Overfitting Phenomenon of Deep Click-Through Rate Prediction Models](https://mlsanity.com/paper/2209.06053/) (2022)

## Built on by (most cited first, 2 of 2)

- [A Probabilistic Framework for Temporal Distribution Generalization in Industry-Scale Recommender Systems](https://mlsanity.com/paper/2511.21032/) (2025)
- [Multimodal Generative Retrieval Model with Staged Pretraining for Food Delivery on Meituan](https://mlsanity.com/paper/2602.06654/) (2026)

## Abstract (arXiv)

ID-based embeddings are widely used in web-scale online recommendation systems. However, their susceptibility to overfitting, particularly due to the long-tail nature of data distributions, often limits training to a single epoch, a phenomenon known as the "one-epoch problem." This challenge has driven research efforts to optimize performance within the first epoch by enhancing convergence speed or feature sparsity. In this study, we introduce a novel two-stage training strategy that incorporates a pre-training phase using a minimal model with contrastive loss, enabling broader data coverage for the embedding system. Our offline experiments demonstrate that multi-epoch training during the pre-training phase does not lead to overfitting, and the resulting embeddings improve online generalization when fine-tuned for more complex downstream recommendation tasks. We deployed the proposed system in live traffic at Pinterest, achieving significant site-wide engagement gains.