# Denoising Pre-Training and Customized Prompt Learning for Efficient Multi-Behavior Sequential Recommendation

arXiv: https://arxiv.org/abs/2408.11372 · HTML: https://mlsanity.com/paper/2408.11372/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 62 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 30 of 44)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [PromptCast: A New Prompt-based Learning Paradigm for Time Series Forecasting](https://mlsanity.com/paper/2210.08964/) (2022)
- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [ReduNet: A White-box Deep Network from the Principle of Maximizing Rate Reduction](https://mlsanity.com/paper/2105.10446/) (2021)
- [Personalized Prompt Learning for Explainable Recommendation](https://mlsanity.com/paper/2202.07371/) (2022)
- [Dynamic Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/2104.07368/) (2021)
- [PinnerFormer: Sequence Modeling for User Representation at Pinterest](https://mlsanity.com/paper/2205.04507/) (2022)
- [Frequency Enhanced Hybrid Attention Network for Sequential Recommendation](https://mlsanity.com/paper/2304.09184/) (2023)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Entropy Law: The Story Behind Data Compression and LLM Performance](https://mlsanity.com/paper/2407.06645/) (2024)
- [Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation](https://mlsanity.com/paper/2207.05584/) (2022)
- [Knowledge-Enhanced Hierarchical Graph Transformer Network for Multi-Behavior Recommendation](https://mlsanity.com/paper/2110.04000/) (2021)
- [Graph Meta Network for Multi-Behavior Recommendation](https://mlsanity.com/paper/2110.03969/) (2021)
- [UP5: Unbiased Foundation Model for Fairness-aware Recommendation](https://mlsanity.com/paper/2305.12090/) (2023)
- [Contrastive Meta Learning with Behavior Multiplicity for Recommendation](https://mlsanity.com/paper/2202.08523/) (2022)
- [MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network](https://mlsanity.com/paper/1905.11013/) (2019)
- [Multiplex Behavioral Relation Learning for Recommendation via Memory Augmented Transformer Network](https://mlsanity.com/paper/2110.04002/) (2021)
- [Prompt Learning for News Recommendation](https://mlsanity.com/paper/2304.05263/) (2023)
- [Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Tasks](https://mlsanity.com/paper/1805.10727/) (2018)
- [FeedRec: News Feed Recommendation with Various User Feedbacks](https://mlsanity.com/paper/2102.04903/) (2021)
- [Multi-view Multi-behavior Contrastive Learning in Recommendation](https://mlsanity.com/paper/2203.10576/) (2022)
- [Multi-Behavior Sequential Recommendation with Temporal Graph Transformer](https://mlsanity.com/paper/2206.02687/) (2022)

## Built on by (most cited first, 3 of 3)

- [Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model](https://mlsanity.com/paper/2410.23994/) (2024)
- [Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts](https://mlsanity.com/paper/2510.25285/) (2025)
- [Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation](https://mlsanity.com/paper/2603.07107/) (2026)

## Abstract (arXiv)

In the realm of recommendation systems, users exhibit a diverse array of behaviors when interacting with items. This phenomenon has spurred research into learning the implicit semantic relationships between these behaviors to enhance recommendation performance. However, these methods often entail high computational complexity. To address concerns regarding efficiency, pre-training presents a viable solution. Its objective is to extract knowledge from extensive pre-training data and fine-tune the model for downstream tasks. Nevertheless, previous pre-training methods have primarily focused on single-behavior data, while multi-behavior data contains significant noise. Additionally, the fully fine-tuning strategy adopted by these methods still imposes a considerable computational burden. In response to this challenge, we propose DPCPL, the first pre-training and prompt-tuning paradigm tailored for Multi-Behavior Sequential Recommendation. Specifically, in the pre-training stage, we commence by proposing a novel Efficient Behavior Miner (EBM) to filter out the noise at multiple time scales, thereby facilitating the comprehension of the contextual semantics of multi-behavior sequences. Subsequently, we propose to tune the pre-trained model in a highly efficient manner with the proposed Customized Prompt Learning (CPL) module, which generates personalized, progressive, and diverse prompts to fully exploit the potential of the pre-trained model effectively. Extensive experiments on three real-world datasets have unequivocally demonstrated that DPCPL not only exhibits high efficiency and effectiveness, requiring minimal parameter adjustments but also surpasses the state-of-the-art performance across a diverse range of downstream tasks.