mlsanityData to 14 Oct 2026

KEEP: An Industrial Pre-Training Framework for Online Recommendation via Knowledge Extraction and Plugging

arXiv 2208.10174 ↗First posted 22 Aug 2022cs.IR cs.AIcited by 11

Not on the map. Why?

Abstract (arXiv)

An industrial recommender system generally presents a hybrid list that contains results from multiple subsystems. In practice, each subsystem is optimized with its own feedback data to avoid the disturbance among different subsystems. However, we argue that such data usage may lead to sub-optimal online performance because of the textitdata sparsity. To alleviate this issue, we propose to extract knowledge from the textitsuper-domain that contains web-scale and long-time impression data, and further assist the online recommendation task (downstream task). To this end, we propose a novel industrial KnowlEdge Extraction and Plugging (KEEP) framework, which is a two-stage framework that consists of 1) a supervised pre-training knowledge extraction module on super-domain, and 2) a plug-in network that incorporates the extracted knowledge into the downstream model. This makes it friendly for incremental training of online recommendation. Moreover, we design an efficient empirical approach for KEEP and introduce our hands-on experience during the implementation of KEEP in a large-scale industrial system. Experiments conducted on two real-world datasets demonstrate that KEEP can achieve promising results. It is notable that KEEP has also been deployed on the display advertising system in Alibaba, bringing a lift of +5.4% CTR and +4.7% RPM.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 22 Aug 2022.

Builds on

30 of 35 references.

References · 30

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2016Overcoming catastrophic forgetting in neural networkscited by 4,787
  3. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  4. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  5. 2017Deep Interest Network for Click-Through Rate Predictioncited by 544
  6. 2018Deep Interest Evolution Network for Click-Through Rate Predictioncited by 338
  7. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  8. 2020Search-based User Interest Modeling with Lifelong Sequential Behavior Data for Click-Through Rate Predictioncited by 177
  9. 2019Behavior Sequence Transformer for E-commerce Recommendation in Alibabacited by 130
  10. 2019Practice on Long Sequential User Behavior Modeling for Click-Through Rate Predictioncited by 118
  11. 2018Learning Tree-based Deep Model for Recommender Systemscited by 99
  12. 2017Online Deep Learning: Learning Deep Neural Networks on the Flycited by 86
18 more
  1. 2018CoNet: Collaborative Cross Networks for Cross-Domain Recommendationcited by 81
  2. 2021One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Predictioncited by 78
  3. 2019DDTCDR: Deep Dual Transfer Cross Domain Recommendationcited by 72
  4. 2017Optimized Cost per Click in Taobao Display Advertisingcited by 57
  5. 2021CauseRec: Counterfactual User Sequence Synthesis for Sequential Recommendationcited by 46
  6. 2020How to Retrain Recommender System? A Sequential Meta-Learning Methodcited by 42
  7. 2019DARec: Deep Domain Adaptation for Cross-Domain Recommendation via Transferring Rating Patternscited by 33
  8. 2018TF-Ranking: Scalable TensorFlow Library for Learning-to-Rankcited by 32
  9. 2018Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Taskscited by 24
  10. 2021FeedRec: News Feed Recommendation with Various User Feedbackscited by 23
  11. 2020PTUM: Pre-training User Model from Unlabeled User Behaviors via Self-supervisioncited by 22
  12. 2021Real Negatives Matter: Continuous Training with Real Negatives for Delayed Feedback Modelingcited by 20
  13. 2022PICASSO: Unleashing the Potential of GPU-centric Training for Wide-and-deep Recommender Systemscited by 17
  14. 2021UPRec: User-Aware Pre-training for Recommender Systemscited by 16
  15. 2021User-specific Adaptive Fine-tuning for Cross-domain Recommendationscited by 9
  16. 2020CAN: Feature Co-Action for Click-Through Rate Predictioncited by 6
  17. 2021Adversarial Gradient Driven Exploration for Deep Click-Through Rate Predictioncited by 4
  18. 2019Deep Cross Networks with Aesthetic Preference for Cross-domain Recommendationcited by 1

Built on by

Cited by year

Citing papers per year: 2023: 4; 2024: 5; 2025: 2.

11 papers.

  1. 2024Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches and Insightscited by 23
  2. 2023Rec4Ad: A Free Lunch to Mitigate Sample Selection Bias for Ads CTR Prediction in Taobaocited by 11
  3. 2023COPR: Consistency-Oriented Pre-Ranking for Online Advertisingcited by 10
  4. 2023Capturing Conversion Rate Fluctuation during Sales Promotions: A Novel Historical Data Reuse Approachcited by 8
  5. 2025See Beyond a Single View: Multi-Attribution Learning Leads to Better Conversion Rate Predictioncited by 5
  6. 2024Improving conversion rate prediction via self-supervised pre-training in online advertisingcited by 2
  7. 2024MMBee: Live Streaming Gift-Sending Recommendations via Multi-Modal Fusion and Behaviour Expansioncited by 2
  8. 2023Impression-Aware Recommender Systemscited by 1
  9. 2024Effective Two-Stage Knowledge Transfer for Multi-Entity Cross-Domain Recommendationcited by 1
  10. 2024DIIT: A Domain-Invariant Information Transfer Method for Industrial Cross-Domain Recommendationcited by 0
  11. 2025PRECTR: A Synergistic Framework for Integrating Personalized Search Relevance Matching and CTR Predictioncited by 0