# Sparse-Interest Network for Sequential Recommendation

arXiv: https://arxiv.org/abs/2102.09267 · HTML: https://mlsanity.com/paper/2102.09267/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Inductive Representation Learning on Large Graphs](https://mlsanity.com/paper/1706.02216/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Graph Convolutional Neural Networks for Web-Scale Recommender Systems](https://mlsanity.com/paper/1806.01973/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Graph Neural Networks for Social Recommendation](https://mlsanity.com/paper/1902.07243/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Dynamic Routing Between Capsules](https://mlsanity.com/paper/1710.09829/) (2017)
- [DeepFM: A Factorization-Machine based Neural Network for CTR Prediction](https://mlsanity.com/paper/1703.04247/) (2017)
- [Deep Interest Network for Click-Through Rate Prediction](https://mlsanity.com/paper/1706.06978/) (2017)
- [Graph U-Nets](https://mlsanity.com/paper/1905.05178/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [A Structured Self-attentive Sentence Embedding](https://mlsanity.com/paper/1703.03130/) (2017)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Multi-Interest Network with Dynamic Routing for Recommendation at Tmall](https://mlsanity.com/paper/1904.08030/) (2019)
- [Learning Disentangled Representations for Recommendation](https://mlsanity.com/paper/1910.14238/) (2019)
- [Fast Matrix Factorization for Online Recommendation with Implicit Feedback](https://mlsanity.com/paper/1708.05024/) (2017)
- [Learning Tree-based Deep Model for Recommender Systems](https://mlsanity.com/paper/1801.02294/) (2018)
- [Reducing Overfitting in Deep Networks by Decorrelating Representations](https://mlsanity.com/paper/1511.06068/) (2015)
- [On Using Very Large Target Vocabulary for Neural Machine Translation](https://mlsanity.com/paper/1412.2007/) (2014)
- [CosRec: 2D Convolutional Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/1908.09972/) (2019)
- [Deep Representation Learning for Social Network Analysis](https://mlsanity.com/paper/1904.08547/) (2019)
- [Graph Convolutional Network for Recommendation with Low-pass Collaborative Filters](https://mlsanity.com/paper/2006.15516/) (2020)
- [Learning to Hash with Graph Neural Networks for Recommender Systems](https://mlsanity.com/paper/2003.01917/) (2020)
- [SDM: Sequential Deep Matching Model for Online Large-scale Recommender System](https://mlsanity.com/paper/1909.00385/) (2019)

## Built on by (most cited first, 30 of 38)

- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation](https://mlsanity.com/paper/2503.22675/) (2025)
- [Edge-Cloud Polarization and Collaboration: A Comprehensive Survey for AI](https://mlsanity.com/paper/2111.06061/) (2021)
- [Device-Cloud Collaborative Learning for Recommendation](https://mlsanity.com/paper/2104.06624/) (2021)
- [Bring Your Own View: Graph Neural Networks for Link Prediction with Personalized Subgraph Selection](https://mlsanity.com/paper/2212.12488/) (2022)
- [GiGaMAE: Generalizable Graph Masked Autoencoder via Collaborative Latent Space Reconstruction](https://mlsanity.com/paper/2308.09663/) (2023)
- [Revisiting Neural Retrieval on Accelerators](https://mlsanity.com/paper/2306.04039/) (2023)
- [Pacer and Runner: Cooperative Learning Framework between Single- and Cross-Domain Sequential Recommendation](https://mlsanity.com/paper/2407.11245/) (2024)
- [Price DOES Matter! Modeling Price and Interest Preferences in Session-based Recommendation](https://mlsanity.com/paper/2205.04181/) (2022)
- [Device-Cloud Collaborative Recommendation via Meta Controller](https://mlsanity.com/paper/2207.03066/) (2022)
- [MobileRec: A Large-Scale Dataset for Mobile Apps Recommendation](https://mlsanity.com/paper/2303.06588/) (2023)
- [Collaborative Graph Neural Networks for Attributed Network Embedding](https://mlsanity.com/paper/2307.11981/) (2023)
- [Click-through Rate Prediction with Auto-Quantized Contrastive Learning](https://mlsanity.com/paper/2109.13921/) (2021)
- [Trinity: Syncretizing Multi-/Long-tail/Long-term Interests All in One](https://mlsanity.com/paper/2402.02842/) (2024)
- [Coarse-to-Fine Knowledge-Enhanced Multi-Interest Learning Framework for Multi-Behavior Recommendation](https://mlsanity.com/paper/2208.01849/) (2022)
- [Implicit Session Contexts for Next-Item Recommendations](https://mlsanity.com/paper/2208.09076/) (2022)
- [Pre-train and Search: Efficient Embedding Table Sharding with Pre-trained Neural Cost Models](https://mlsanity.com/paper/2305.01868/) (2023)
- [Ada-Retrieval: An Adaptive Multi-Round Retrieval Paradigm for Sequential Recommendations](https://mlsanity.com/paper/2401.06633/) (2024)
- [Revisiting Self-attention for Cross-domain Sequential Recommendation](https://mlsanity.com/paper/2505.21811/) (2025)
- [Deep Stable Multi-Interest Learning for Out-of-distribution Sequential Recommendation](https://mlsanity.com/paper/2304.05615/) (2023)
- [Leave No One Behind: Online Self-Supervised Self-Distillation for Sequential Recommendation](https://mlsanity.com/paper/2404.07219/) (2024)
- [Capturing Popularity Trends: A Simplistic Non-Personalized Approach for Enhanced Item Recommendation](https://mlsanity.com/paper/2308.08799/) (2023)
- [Group-Aware Interest Disentangled Dual-Training for Personalized Recommendation](https://mlsanity.com/paper/2311.09577/) (2023)
- [BiVRec: Bidirectional View-based Multimodal Sequential Recommendation](https://mlsanity.com/paper/2402.17334/) (2024)
- [Recommender Transformers with Behavior Pathways](https://mlsanity.com/paper/2206.06804/) (2022)
- [Improving Multi-Interest Network with Stable Learning](https://mlsanity.com/paper/2207.07910/) (2022)
- [Hierarchical Multi-Interest Co-Network For Coarse-Grained Ranking](https://mlsanity.com/paper/2210.10547/) (2022)
- [Methodologies for Improving Modern Industrial Recommender Systems](https://mlsanity.com/paper/2308.01204/) (2023)
- [Personalized Programming Guidance based on Deep Programming Learning Style Capturing](https://mlsanity.com/paper/2403.14638/) (2024)
- [Deep Uncertainty-Based Explore for Index Construction and Retrieval in Recommendation System](https://mlsanity.com/paper/2408.00799/) (2024)

## Abstract (arXiv)

Recent methods in sequential recommendation focus on learning an overall embedding vector from a user's behavior sequence for the next-item recommendation. However, from empirical analysis, we discovered that a user's behavior sequence often contains multiple conceptually distinct items, while a unified embedding vector is primarily affected by one's most recent frequent actions. Thus, it may fail to infer the next preferred item if conceptually similar items are not dominant in recent interactions. To this end, an alternative solution is to represent each user with multiple embedding vectors encoding different aspects of the user's intentions. Nevertheless, recent work on multi-interest embedding usually considers a small number of concepts discovered via clustering, which may not be comparable to the large pool of item categories in real systems. It is a non-trivial task to effectively model a large number of diverse conceptual prototypes, as items are often not conceptually well clustered in fine granularity. Besides, an individual usually interacts with only a sparse set of concepts. In light of this, we propose a novel Sparse Interest NEtwork (SINE) for sequential recommendation. Our sparse-interest module can adaptively infer a sparse set of concepts for each user from the large concept pool and output multiple embeddings accordingly. Given multiple interest embeddings, we develop an interest aggregation module to actively predict the user's current intention and then use it to explicitly model multiple interests for next-item prediction. Empirical results on several public benchmark datasets and one large-scale industrial dataset demonstrate that SINE can achieve substantial improvement over state-of-the-art methods.