# Multi-Grained Patch Training for Efficient LLM-based Recommendation

arXiv: https://arxiv.org/abs/2501.15087 · HTML: https://mlsanity.com/paper/2501.15087/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jan 2025.

## Built on by (most cited first, 5 of 5)

- [HiSAC: Hierarchical Sparse Activation Compression for Ultra-long Sequence Modeling in Recommenders](https://mlsanity.com/paper/2602.21009/) (2026)
- [ItemRAG: Item-Based Retrieval-Augmented Generation for LLM-Based Recommendation](https://mlsanity.com/paper/2511.15141/) (2025)
- [Sensory-Aware Sequential Recommendation via Review-Distilled Representations](https://mlsanity.com/paper/2603.02709/) (2026)
- [MERIT: Mitigating Exposure Bias in Generative XMC for User-Interest Propensity Modeling](https://mlsanity.com/paper/2608.28931/) (2026)
- [KuaFu: Compressing Long User Behavior into Understanding at Billion Scale](https://mlsanity.com/paper/2609.31045/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) have emerged as a new paradigm for recommendation by converting interacted item history into language modeling. However, constrained by the limited context length of LLMs, existing approaches have to truncate item history in the prompt, focusing only on recent interactions and sacrificing the ability to model long-term history. To enable LLMs to model long histories, we pursue a concise embedding representation for items and sessions. In the LLM embedding space, we construct an item's embedding by aggregating its textual token embeddings; similarly, we construct a session's embedding by aggregating its item embeddings. While efficient, this way poses two challenges since it ignores the temporal significance of user interactions and LLMs do not natively interpret our custom embeddings. To overcome these, we propose PatchRec, a multi-grained patch training method consisting of two stages: (1) Patch Pre-training, which familiarizes LLMs with aggregated embeddings -- patches, and (2) Patch Fine-tuning, which enables LLMs to capture time-aware significance in interaction history. Extensive experiments show that PatchRec effectively models longer behavior histories with improved efficiency. This work facilitates the practical use of LLMs for modeling long behavior histories. Codes are available at https://github.com/ljy0ustc/PatchRec.