# MPFormer: Adaptive Framework for Industrial Multi-Task Personalized Sequential Retriever

arXiv: https://arxiv.org/abs/2508.20400 · HTML: https://mlsanity.com/paper/2508.20400/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 30 references that are in the map.
- First posted on arXiv 28 Aug 2025.

## Builds on (most cited first, 21 of 21)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Decision Transformer: Reinforcement Learning via Sequence Modeling](https://mlsanity.com/paper/2106.01345/) (2021)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [Multi-Task Learning Using Uncertainty to Weigh Losses for Scene Geometry and Semantics](https://mlsanity.com/paper/1705.07115/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [GradNorm: Gradient Normalization for Adaptive Loss Balancing in Deep Multitask Networks](https://mlsanity.com/paper/1711.02257/) (2017)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Embedding-based Retrieval in Facebook Search](https://mlsanity.com/paper/2006.11632/) (2020)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [Pareto Set Learning for Neural Multi-objective Combinatorial Optimization](https://mlsanity.com/paper/2203.15386/) (2022)
- [SDM: Sequential Deep Matching Model for Online Large-scale Recommender System](https://mlsanity.com/paper/1909.00385/) (2019)
- [OneRec Technical Report](https://mlsanity.com/paper/2506.13695/) (2025)
- [HoME: Hierarchy of Multi-Gate Experts for Multi-Task Learning at Kuaishou](https://mlsanity.com/paper/2408.05430/) (2024)
- [Multi-Objective Personalized Product Retrieval in Taobao Search](https://mlsanity.com/paper/2210.04170/) (2022)
- [Mixture of Virtual-Kernel Experts for Multi-Objective User Profile Modeling](https://mlsanity.com/paper/2106.07356/) (2021)
- [Deep Pareto Reinforcement Learning for Multi-Objective Recommender Systems](https://mlsanity.com/paper/2407.03580/) (2024)
- [Everyone's Preference Changes Differently: Weighted Multi-Interest Retrieval Model](https://mlsanity.com/paper/2207.06652/) (2022)
- [Distillation based Multi-task Learning: A Candidate Generation Model for Improving Reading Duration](https://mlsanity.com/paper/2102.07142/) (2021)

## Abstract (arXiv)

Modern industrial recommendation systems encounter a core challenge of multi-stage optimization misalignment: a significant semantic gap exists between the multi-objective optimization paradigm widely used in the ranking phase and the single-objective modeling in the retrieve phase. Although the mainstream industry solution achieves multi-objective coverage through parallel multi-path single-objective retrieval, this approach leads to linear growth of training and serving resources with the number of objectives and has inherent limitations in handling loosely coupled objectives. This paper proposes the MPFormer, a dynamic multi-task Transformer framework, which systematically addresses the aforementioned issues through three innovative mechanisms. First, an objective-conditioned transformer that jointly encodes user behavior sequences and multi-task semantics through learnable attention modulation; second, personalized target weights are introduced to achieve dynamic adjustment of retrieval results; finally, user personalization information is incorporated into token representations and the Transformer structure to further enhance the model's representation ability. This framework has been successfully integrated into Kuaishou short video recommendation system, stably serving over 400 million daily active users. It significantly improves user daily engagement and system operational efficiency. Practical deployment verification shows that, compared with traditional solutions, it effectively optimizes the iterative paradigm of multi-objective retrieval while maintaining service response speed, providing a scalable multi-objective solution for industrial recommendation systems.