# An Incremental Learning framework for Large-scale CTR Prediction

arXiv: https://arxiv.org/abs/2209.00458 · HTML: https://mlsanity.com/paper/2209.00458/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 1 Sep 2022.

## Builds on (most cited first, 7 of 7)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Wide & Deep Learning for Recommender Systems](https://mlsanity.com/paper/1606.07792/) (2016)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Using Pre-Training Can Improve Model Robustness and Uncertainty](https://mlsanity.com/paper/1901.09960/) (2019)

## Built on by (most cited first, 5 of 5)

- [Multi-Epoch learning with Data Augmentation for Deep Click-Through Rate Prediction](https://mlsanity.com/paper/2407.01607/) (2024)
- [Retrieval and Distill: A Temporal Data Shift-Free Paradigm for Online Recommendation System](https://mlsanity.com/paper/2404.15678/) (2024)
- [DIIT: A Domain-Invariant Information Transfer Method for Industrial Cross-Domain Recommendation](https://mlsanity.com/paper/2410.10835/) (2024)
- [Feature Staleness Aware Incremental Learning for CTR Prediction](https://mlsanity.com/paper/2505.02844/) (2025)
- [MEGG: Replay via Maximally Extreme GGscore in Incremental Learning for Neural Recommendation Models](https://mlsanity.com/paper/2509.07319/) (2025)

## Abstract (arXiv)

In this work we introduce an incremental learning framework for Click-Through-Rate (CTR) prediction and demonstrate its effectiveness for Taboola's massive-scale recommendation service. Our approach enables rapid capture of emerging trends through warm-starting from previously deployed models and fine tuning on "fresh" data only. Past knowledge is maintained via a teacher-student paradigm, where the teacher acts as a distillation technique, mitigating the catastrophic forgetting phenomenon. Our incremental learning framework enables significantly faster training and deployment cycles (x12 speedup). We demonstrate a consistent Revenue Per Mille (RPM) lift over multiple traffic segments and a significant CTR increase on newly introduced items.