# LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systems

arXiv: https://arxiv.org/abs/2411.01537 · HTML: https://mlsanity.com/paper/2411.01537/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2024.

## Built on by (most cited first, 21 of 21)

- [Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancement](https://mlsanity.com/paper/2412.01282/) (2024)
- [HiD-VAE: Interpretable Generative Recommendation via Hierarchical and Disentangled Semantic IDs](https://mlsanity.com/paper/2508.04618/) (2025)
- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [ECKGBench: Benchmarking Large Language Models in E-commerce Leveraging Knowledge Graph](https://mlsanity.com/paper/2503.15990/) (2025)
- [Temporal Linear Item-Item Model for Sequential Recommendation](https://mlsanity.com/paper/2412.07382/) (2024)
- [Exploring Recommender System Evaluation: A Multi-Modal User Agent Framework for A/B Testing](https://mlsanity.com/paper/2601.04554/) (2026)
- [Revisiting Funnel Transformers for Modern LLM Architectures with Comprehensive Ablations in Training and Inference Configurations](https://mlsanity.com/paper/2504.02877/) (2025)
- [Revisiting Self-attention for Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2505.21811/) (2025)
- [Multi-task Offline Reinforcement Learning for Online Advertising in Recommender Systems](https://mlsanity.com/paper/2506.23090/) (2025)
- [Continual Recommender Systems](https://mlsanity.com/paper/2507.03861/) (2025)
- [Empowering Large Language Model for Sequential Recommendation via Multimodal Embeddings and Semantic IDs](https://mlsanity.com/paper/2509.02017/) (2025)
- [Graph Federated Learning for Personalized Privacy Recommendation](https://mlsanity.com/paper/2508.06208/) (2025)
- [MTRec: Learning to Align with User Preferences via Mental Reward Models](https://mlsanity.com/paper/2509.22807/) (2025)
- [Multi-Perspective Attention Mechanism for Bias-Aware Sequential Recommendation](https://mlsanity.com/paper/2504.05323/) (2025)
- [EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation](https://mlsanity.com/paper/2508.16170/) (2025)
- [PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation](https://mlsanity.com/paper/2601.16556/) (2026)
- [TFPS: A Temporal Filtration-enhanced Positive Sample Set Construction Method for Implicit Collaborative Filtering](https://mlsanity.com/paper/2602.22521/) (2026)
- [ID and Graph View Contrastive Learning with Multi-View Attention Fusion for Sequential Recommendation](https://mlsanity.com/paper/2604.14114/) (2026)
- [FreeScale: Distributed Training for Sequence Recommendation Models with Minimal Scaling Cost](https://mlsanity.com/paper/2604.24073/) (2026)
- [GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation](https://mlsanity.com/paper/2608.16073/) (2026)
- [Dual-Interest Sequential Product Recommendation With Multi-Granular SSM](https://mlsanity.com/paper/2609.21548/) (2026)

## Abstract (arXiv)

Transformer models have achieved remarkable success in sequential recommender systems (SRSs). However, computing the attention matrix in traditional dot-product attention mechanisms results in a quadratic complexity with sequence lengths, leading to high computational costs for long-term sequential recommendation. Motivated by the above observation, we propose a novel L2-Normalized Linear Attention for the Transformer-based Sequential Recommender Systems (LinRec), which theoretically improves efficiency while preserving the learning capabilities of the traditional dot-product attention. Specifically, by thoroughly examining the equivalence conditions of efficient attention mechanisms, we show that LinRec possesses linear complexity while preserving the property of attention mechanisms. In addition, we reveal its latent efficiency properties by interpreting the proposed LinRec mechanism through a statistical lens. Extensive experiments are conducted based on two public benchmark datasets, demonstrating that the combination of LinRec and Transformer models achieves comparable or even superior performance than state-of-the-art Transformer-based SRS models while significantly improving time and memory efficiency.