# SPAR: Personalized Content-Based Recommendation via Long Engagement Attention

arXiv: https://arxiv.org/abs/2402.10555 · HTML: https://mlsanity.com/paper/2402.10555/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 60 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 21 of 21)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering](https://mlsanity.com/paper/2007.01282/) (2020)
- [Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration](https://mlsanity.com/paper/2306.09093/) (2023)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [M6-Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems](https://mlsanity.com/paper/2205.08084/) (2022)
- [LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructions](https://mlsanity.com/paper/2304.14402/) (2023)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [NPA: Neural News Recommendation with Personalized Attention](https://mlsanity.com/paper/1907.05559/) (2019)
- [Empowering News Recommendation with Pre-trained Language Models](https://mlsanity.com/paper/2104.07413/) (2021)
- [Neural News Recommendation with Attentive Multi-View Learning](https://mlsanity.com/paper/1907.05576/) (2019)
- [HieRec: Hierarchical User Interest Modeling for Personalized News Recommendation](https://mlsanity.com/paper/2106.04408/) (2021)
- [Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension](https://mlsanity.com/paper/2005.08056/) (2020)
- [UniTRec: A Unified Text-to-Text Transformer and Joint Contrastive Learning Framework for Text-based Recommendation](https://mlsanity.com/paper/2305.15756/) (2023)
- [NewsBERT: Distilling Pre-trained Language Model for Intelligent News Application](https://mlsanity.com/paper/2102.04887/) (2021)
- [News Recommendation with Candidate-aware User Modeling](https://mlsanity.com/paper/2204.04726/) (2022)
- [Text Matching Improves Sequential Recommendation by Reducing Popularity Biases](https://mlsanity.com/paper/2308.14029/) (2023)
- [NewsRecLib: A PyTorch-Lightning Library for Neural News Recommendation](https://mlsanity.com/paper/2310.01146/) (2023)

## Built on by (most cited first, 6 of 6)

- [In Prospect and Retrospect: Reflective Memory Management for Long-term Personalized Dialogue Agents](https://mlsanity.com/paper/2503.08026/) (2025)
- [Personalized LLM Decoding via Contrasting Personal Preference](https://mlsanity.com/paper/2506.12109/) (2025)
- [Molar: Multimodal LLMs with Collaborative Filtering Alignment for Enhanced Sequential Recommendation](https://mlsanity.com/paper/2412.18176/) (2024)
- [A Survey on LLM-based News Recommender Systems](https://mlsanity.com/paper/2502.09797/) (2025)
- [Using LLMs to Capture Users' Temporal Context for Recommendation](https://mlsanity.com/paper/2508.08512/) (2025)
- [LLM-Assisted Reranking to Operationalize Nuanced Objectives in Recommender Systems](https://mlsanity.com/paper/2606.02883/) (2026)

## Abstract (arXiv)

Leveraging users' long engagement histories is essential for personalized content recommendations. The success of pretrained language models (PLMs) in NLP has led to their use in encoding user histories and candidate items, framing content recommendations as textual semantic matching tasks. However, existing works still struggle with processing very long user historical text and insufficient user-item interaction. In this paper, we introduce a content-based recommendation framework, SPAR, which effectively tackles the challenges of holistic user interest extraction from the long user engagement history. It achieves so by leveraging PLM, poly-attention layers and attention sparsity mechanisms to encode user's history in a session-based manner. The user and item side features are sufficiently fused for engagement prediction while maintaining standalone representations for both sides, which is efficient for practical model deployment. Moreover, we enhance user profiling by exploiting large language model (LLM) to extract global interests from user engagement history. Extensive experiments on two benchmark datasets demonstrate that our framework outperforms existing state-of-the-art (SoTA) methods.