# Soft Curriculum Learning for Optimizing Fresh and Generalized Recommendations

arXiv: https://arxiv.org/abs/2609.35783 · HTML: https://mlsanity.com/paper/2609.35783/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 11 Aug 2026.

## Builds on (most cited first, 7 of 7)

- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [Automatic Curriculum Learning through Value Disagreement](https://mlsanity.com/paper/2006.09641/) (2020)
- [Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks](https://mlsanity.com/paper/1611.06204/) (2016)
- [Efficient Document Re-Ranking for Transformers by Precomputing Term Representations](https://mlsanity.com/paper/2004.14255/) (2020)
- [Curriculum Learning for Data-Efficient Vision-Language Alignment](https://mlsanity.com/paper/2207.14525/) (2022)
- [Adaptive Hardness Negative Sampling for Collaborative Filtering](https://mlsanity.com/paper/2401.05191/) (2024)

## Abstract (arXiv)

Large-scale recommender systems, particularly short-form video platforms, are often bottlenecked by massive popularity feedback loops. In such environments, as models recommend popular items, they generate an overwhelming amount of skewed training data for "head" items. This creates a self-reinforcing cycle where retrieval and ranking models memorize "head" item patterns at the expense of generalizing across the vast "tail" of the catalogue. While Curriculum Learning (CL) offers a powerful mechanism to break this feedback loop by systematically exposing models to progressively more difficult and less frequent examples, its adoption in industrial recommendation has been hampered by hardware utilization inefficiencies or the needs for complicated pre-processing techniques because dynamic data rejection algorithms tend to starve hardware accelearators (TPUs/GPUs) by becoming largely CPU-bound. In this work, we introduce a scalable Soft Curriculum Learning framework designed specifically for continuous training setups within industry-scale retrieval and ranking models. By utilizing loss annealing and in-graph weight adjustments rather than rigid data filtering, we break the popularity feedback and enable dynamic curriculum pacing without sacrificing system throughput. We demonstrate empirical evidence through applications across sequence-based retrieval models (such as SASRec), two-tower retrieval models, and large-scale continuous ranking models. Online A/B tests on our short-video platform demonstrate substantial lifts in both overall user satisfaction and fresh content consumption, all without degrading model throughput.