# A Practical Incremental Method to Train Deep CTR Models

arXiv: https://arxiv.org/abs/2009.02147 · HTML: https://mlsanity.com/paper/2009.02147/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [ScaleFreeCTR: MixCache-based Distributed Training System for CTR Models with Huge Embedding Table](https://mlsanity.com/paper/2104.08542/) (2021)
- [Future Gradient Descent for Adapting the Temporal Shifting Data Distribution in Online Recommendation Systems](https://mlsanity.com/paper/2209.01143/) (2022)
- [Efficient Multimodal Streaming Recommendation via Expandable Side Mixture-of-Experts](https://mlsanity.com/paper/2508.05993/) (2025)
- [Always Strengthen Your Strengths: A Drift-Aware Incremental Learning Framework for CTR Prediction](https://mlsanity.com/paper/2304.09062/) (2023)
- [CPMR: Context-Aware Incremental Sequential Recommendation with Pseudo-Multi-Task Learning](https://mlsanity.com/paper/2309.04802/) (2023)
- [On the Adaptation to Concept Drift for CTR Prediction](https://mlsanity.com/paper/2204.05101/) (2022)
- [KDSM: An uplift modeling framework based on knowledge distillation and sample matching](https://mlsanity.com/paper/2303.02980/) (2023)
- [FINED: Feed Instance-Wise Information Need with Essential and Disentangled Parametric Knowledge from the Past](https://mlsanity.com/paper/2406.00012/) (2024)
- [Feature Staleness Aware Incremental Learning for CTR Prediction](https://mlsanity.com/paper/2505.02844/) (2025)
- [MEGG: Replay via Maximally Extreme GGscore in Incremental Learning for Neural Recommendation Models](https://mlsanity.com/paper/2509.07319/) (2025)

## Abstract (arXiv)

Deep learning models in recommender systems are usually trained in the batch mode, namely iteratively trained on a fixed-size window of training data. Such batch mode training of deep learning models suffers from low training efficiency, which may lead to performance degradation when the model is not produced on time. To tackle this issue, incremental learning is proposed and has received much attention recently. Incremental learning has great potential in recommender systems, as two consecutive window of training data overlap most of the volume. It aims to update the model incrementally with only the newly incoming samples from the timestamp when the model is updated last time, which is much more efficient than the batch mode training. However, most of the incremental learning methods focus on the research area of image recognition where new tasks or classes are learned over time. In this work, we introduce a practical incremental method to train deep CTR models, which consists of three decoupled modules (namely, data, feature and model module). Our method can achieve comparable performance to the conventional batch mode training with much better training efficiency. We conduct extensive experiments on a public benchmark and a private dataset to demonstrate the effectiveness of our proposed method.