# A Population-to-individual Tuning Framework for Adapting Pretrained LM to On-device User Intent Prediction

arXiv: https://arxiv.org/abs/2408.09815 · HTML: https://mlsanity.com/paper/2408.09815/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 19 Aug 2024.

## Builds on (most cited first, 27 of 27)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [The Power of Scale for Parameter-Efficient Prompt Tuning](https://mlsanity.com/paper/2104.08691/) (2021)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [One Fits All:Power General Time Series Analysis by Pretrained LM](https://mlsanity.com/paper/2302.11939/) (2023)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)](https://mlsanity.com/paper/2203.13366/) (2022)
- [TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation](https://mlsanity.com/paper/2305.00447/) (2023)
- [Large Language Models Empowered Agent-based Modeling and Simulation: A Survey and Perspectives](https://mlsanity.com/paper/2312.11970/) (2023)
- [Unlearn What You Want to Forget: Efficient Unlearning for LLMs](https://mlsanity.com/paper/2310.20150/) (2023)
- [Recommendation as Instruction Following: A Large Language Model Empowered Recommendation Approach](https://mlsanity.com/paper/2305.07001/) (2023)
- [UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting](https://mlsanity.com/paper/2310.09751/) (2023)
- [Pre-training Enhanced Spatial-temporal Graph Neural Network for Multivariate Time Series Forecasting](https://mlsanity.com/paper/2206.09113/) (2022)
- [UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction](https://mlsanity.com/paper/2402.11838/) (2024)
- [DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization](https://mlsanity.com/paper/2209.05227/) (2022)
- [Device-Cloud Collaborative Learning for Recommendation](https://mlsanity.com/paper/2104.06624/) (2021)
- [Using Sequences of Life-events to Predict Human Lives](https://mlsanity.com/paper/2306.03009/) (2023)
- [Learning to Simulate Daily Activities via Modeling Dynamic Human Needs](https://mlsanity.com/paper/2302.10897/) (2023)
- [Session-based Recommendation with Hypergraph Attention Networks](https://mlsanity.com/paper/2112.14266/) (2021)
- [Comprehensive Fair Meta-learned Recommender System](https://mlsanity.com/paper/2206.04789/) (2022)
- [IART: Intent-aware Response Ranking with Transformers in Information-seeking Conversation Systems](https://mlsanity.com/paper/2002.00571/) (2020)
- [Efficient On-Device Session-Based Recommendation](https://mlsanity.com/paper/2209.13422/) (2022)
- [Alleviating Video-Length Effect for Micro-video Recommendation](https://mlsanity.com/paper/2308.14276/) (2023)
- [Preliminary Study on Incremental Learning for Large Language Model-based Recommender Systems](https://mlsanity.com/paper/2312.15599/) (2023)
- [DC-CCL: Device-Cloud Collaborative Controlled Learning for Large Vision Models](https://mlsanity.com/paper/2303.10361/) (2023)
- [Intent-aware Ranking Ensemble for Personalized Recommendation](https://mlsanity.com/paper/2304.07450/) (2023)
- [Meta-Learning with Adaptive Weighted Loss for Imbalanced Cold-Start Recommendation](https://mlsanity.com/paper/2302.14640/) (2023)

## Built on by (most cited first, 6 of 6)

- [Split Adaptation for Pre-trained Vision Transformers](https://mlsanity.com/paper/2503.00441/) (2025)
- [BehaveGPT: A Foundation Model for Large-scale User Behavior Modeling](https://mlsanity.com/paper/2505.17631/) (2025)
- [Tuning Language Models for Robust Prediction of Diverse User Behaviors](https://mlsanity.com/paper/2505.17682/) (2025)
- [Large language model as user daily behavior data generator: balancing population diversity and individual personality](https://mlsanity.com/paper/2505.17615/) (2025)
- [IntPro: A Proxy Agent for Context-Aware Intent Understanding via Retrieval-conditioned Inference](https://mlsanity.com/paper/2603.03325/) (2026)
- [LLMs Reading the Rhythms of Daily Life: Aligned Understanding for Behavior Prediction and Generation](https://mlsanity.com/paper/2604.23578/) (2026)

## Abstract (arXiv)

Mobile devices, especially smartphones, can support rich functions and have developed into indispensable tools in daily life. With the rise of generative AI services, smartphones can potentially transform into personalized assistants, anticipating user needs and scheduling services accordingly. Predicting user intents on smartphones, and reflecting anticipated activities based on past interactions and context, remains a pivotal step towards this vision. Existing research predominantly focuses on specific domains, neglecting the challenge of modeling diverse event sequences across dynamic contexts. Leveraging pre-trained language models (PLMs) offers a promising avenue, yet adapting PLMs to on-device user intent prediction presents significant challenges. To address these challenges, we propose PITuning, a Population-to-Individual Tuning framework. PITuning enhances common pattern extraction through dynamic event-to-intent transition modeling and addresses long-tailed preferences via adaptive unlearning strategies. Experimental results on real-world datasets demonstrate PITuning's superior intent prediction performance, highlighting its ability to capture long-tailed preferences and its practicality for on-device prediction scenarios.