# Enhancing CTR Prediction through Sequential Recommendation Pre-training: Introducing the SRP4CTR Framework

arXiv: https://arxiv.org/abs/2407.19658 · HTML: https://mlsanity.com/paper/2407.19658/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jul 2024.

## Built on by (most cited first, 7 of 7)

- [Hierarchical LoRA MoE for Efficient CTR Model Scaling](https://mlsanity.com/paper/2510.10432/) (2025)
- [GR-LLMs: Recent Advances in Generative Recommendation Based on Large Language Models](https://mlsanity.com/paper/2507.06507/) (2025)
- [RecIS: Sparse to Dense, A Unified Training Framework for Recommendation Models](https://mlsanity.com/paper/2509.20883/) (2025)
- [Scaled Supervision is an Implicit Lipschitz Regularizer](https://mlsanity.com/paper/2503.14813/) (2025)
- [Target-Aware Early Stage Ranking](https://mlsanity.com/paper/2511.21095/) (2025)
- [Abacus: Self-Supervised Event Counting-Aligned Distributional Pretraining for Sequential User Modeling](https://mlsanity.com/paper/2512.16581/) (2025)
- [Conversion rate prediction in online advertising: modeling techniques, performance evaluation and future directions](https://mlsanity.com/paper/2512.01171/) (2025)

## Abstract (arXiv)

Understanding user interests is crucial for Click-Through Rate (CTR) prediction tasks. In sequential recommendation, pre-training from user historical behaviors through self-supervised learning can better comprehend user dynamic preferences, presenting the potential for direct integration with CTR tasks. Previous methods have integrated pre-trained models into downstream tasks with the sole purpose of extracting semantic information or well-represented user features, which are then incorporated as new features. However, these approaches tend to ignore the additional inference costs to the downstream tasks, and they do not consider how to transfer the effective information from the pre-trained models for specific estimated items in CTR prediction. In this paper, we propose a Sequential Recommendation Pre-training framework for CTR prediction (SRP4CTR) to tackle the above problems. Initially, we discuss the impact of introducing pre-trained models on inference costs. Subsequently, we introduced a pre-trained method to encode sequence side information concurrently.During the fine-tuning process, we incorporate a cross-attention block to establish a bridge between estimated items and the pre-trained model at a low cost. Moreover, we develop a querying transformer technique to facilitate the knowledge transfer from the pre-trained model to industrial CTR models. Offline and online experiments show that our method outperforms previous baseline models.