# Decoupled Side Information Fusion for Sequential Recommendation

arXiv: https://arxiv.org/abs/2204.11046 · HTML: https://mlsanity.com/paper/2204.11046/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 38 references that are in the map.
- First posted on arXiv 23 Apr 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Rethinking Positional Encoding in Language Pre-training](https://mlsanity.com/paper/2006.15595/) (2020)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [A Simple Convolutional Generative Network for Next Item Recommendation](https://mlsanity.com/paper/1808.05163/) (2018)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Sequential Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2106.14226/) (2021)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation](https://mlsanity.com/paper/2103.03578/) (2021)
- [CosRec: 2D Convolutional Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/1908.09972/) (2019)
- [A Simple and Effective Positional Encoding for Transformers](https://mlsanity.com/paper/2104.08698/) (2021)
- [Context-aware Sequential Recommendation](https://mlsanity.com/paper/1609.05787/) (2016)
- [A Case Study on Sampling Strategies for Evaluating Neural Sequential Item Recommendation Models](https://mlsanity.com/paper/2107.13045/) (2021)
- [Lightweight Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/2108.11333/) (2021)
- [Using Temporal Data for Making Recommendations](https://mlsanity.com/paper/1301.2320/) (2013)
- [Consistency-Aware Recommendation for User-Generated ItemList Continuation](https://mlsanity.com/paper/1912.13031/) (2019)
- [Attribute-aware Diversification for Sequential Recommendations](https://mlsanity.com/paper/2008.00783/) (2020)

## Built on by (most cited first, 30 of 30)

- [LLMRec: Benchmarking Large Language Models on Recommendation Task](https://mlsanity.com/paper/2308.12241/) (2023)
- [A Survey on User Behavior Modeling in Recommender Systems](https://mlsanity.com/paper/2302.11087/) (2023)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [Knowledge Prompt-tuning for Sequential Recommendation](https://mlsanity.com/paper/2308.08459/) (2023)
- [Text Matching Improves Sequential Recommendation by Reducing Popularity Biases](https://mlsanity.com/paper/2308.14029/) (2023)
- [Temporal Interest Network for User Response Prediction](https://mlsanity.com/paper/2308.08487/) (2023)
- [Robust Image Semantic Coding with Learnable CSI Fusion Masking over MIMO Fading Channels](https://mlsanity.com/paper/2406.07389/) (2024)
- [Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation](https://mlsanity.com/paper/2403.16427/) (2024)
- [Dual Contrastive Transformer for Hierarchical Preference Modeling in Sequential Recommendation](https://mlsanity.com/paper/2410.22790/) (2024)
- [NLGR: Utilizing Neighbor Lists for Generative Rerank in Personalized Recommendation Systems](https://mlsanity.com/paper/2502.06097/) (2025)
- [Integrating Large Language Models with Graphical Session-Based Recommendation](https://mlsanity.com/paper/2402.16539/) (2024)
- [AutoAttention: Automatic Field Pair Selection for Attention in User Behavior Modeling](https://mlsanity.com/paper/2210.15154/) (2022)
- [Deep Stable Multi-Interest Learning for Out-of-distribution Sequential Recommendation](https://mlsanity.com/paper/2304.05615/) (2023)
- [KGIF: Optimizing Relation-Aware Recommendations with Knowledge Graph Information Fusion](https://mlsanity.com/paper/2501.04161/) (2025)
- [Efficient and Effective Query Context-Aware Learning-to-Rank Model for Sequential Recommendation](https://mlsanity.com/paper/2507.03789/) (2025)
- [Exploiting Inter-Session Information with Frequency-enhanced Dual-Path Networks for Sequential Recommendation](https://mlsanity.com/paper/2511.06285/) (2025)
- [Multiple Key-value Strategy in Recommendation Systems Incorporating Large Language Model](https://mlsanity.com/paper/2310.16409/) (2023)
- [BiVRec: Bidirectional View-based Multimodal Sequential Recommendation](https://mlsanity.com/paper/2402.17334/) (2024)
- [Attention-based sequential recommendation system using multimodal data](https://mlsanity.com/paper/2405.17959/) (2024)
- [HoMer: Addressing Heterogeneities by Modeling Sequential and Set-wise Contexts for CTR Prediction](https://mlsanity.com/paper/2510.11100/) (2025)
- [Feature-augmented Machine Reading Comprehension with Auxiliary Tasks](https://mlsanity.com/paper/2211.09438/) (2022)
- [A GNN Model with Adaptive Weights for Session-Based Recommendation Systems](https://mlsanity.com/paper/2408.05051/) (2024)
- [Modeling and Analyzing the Influence of Non-Item Pages on Sequential Next-Item Prediction](https://mlsanity.com/paper/2408.15953/) (2024)
- [Token-Efficient Item Representation via Images for LLM Recommender Systems](https://mlsanity.com/paper/2503.06238/) (2025)
- [GemiRec: Interest Quantization and Generation for Multi-Interest Recommendation](https://mlsanity.com/paper/2510.14626/) (2025)
- [Sensory-Aware Sequential Recommendation via Review-Distilled Representations](https://mlsanity.com/paper/2603.02709/) (2026)
- [CAMMSR: Category-Guided Attentive Mixture of Experts for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.04320/) (2026)
- [Rethinking Convolutional Networks for Attribute-Aware Sequential Recommendation](https://mlsanity.com/paper/2605.04723/) (2026)
- [Keeping the Index Open: The Recommendation-Side Cost of Shared Search and Recommendation](https://mlsanity.com/paper/2608.24079/) (2026)
- [RouteRec: Behavior-Guided Sparse Routing for Sequential Recommendation](https://mlsanity.com/paper/2609.39007/) (2026)

## Abstract (arXiv)

Side information fusion for sequential recommendation (SR) aims to effectively leverage various side information to enhance the performance of next-item prediction. Most state-of-the-art methods build on self-attention networks and focus on exploring various solutions to integrate the item embedding and side information embeddings before the attention layer. However, our analysis shows that the early integration of various types of embeddings limits the expressiveness of attention matrices due to a rank bottleneck and constrains the flexibility of gradients. Also, it involves mixed correlations among the different heterogeneous information resources, which brings extra disturbance to attention calculation. Motivated by this, we propose Decoupled Side Information Fusion for Sequential Recommendation (DIF-SR), which moves the side information from the input to the attention layer and decouples the attention calculation of various side information and item representation. We theoretically and empirically show that the proposed solution allows higher-rank attention matrices and flexible gradients to enhance the modeling capacity of side information fusion. Also, auxiliary attribute predictors are proposed to further activate the beneficial interaction between side information and item representation learning. Extensive experiments on four real-world datasets demonstrate that our proposed solution stably outperforms state-of-the-art SR models. Further studies show that our proposed solution can be readily incorporated into current attention-based SR models and significantly boost performance. Our source code is available at https://github.com/AIM-SE/DIF-SR.