# Sequential Recommendation with Auxiliary Item Relationships via Multi-Relational Transformer

arXiv: https://arxiv.org/abs/2210.13572 · HTML: https://mlsanity.com/paper/2210.13572/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 50 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Embedding Entities and Relations for Learning and Inference in Knowledge Bases](https://mlsanity.com/paper/1412.6575/) (2014)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [A Survey on Knowledge Graphs: Representation, Acquisition and Applications](https://mlsanity.com/paper/2002.00388/) (2020)
- [GraphSAINT: Graph Sampling Based Inductive Learning Method](https://mlsanity.com/paper/1907.04931/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [RippleNet: Propagating User Preferences on the Knowledge Graph for Recommender Systems](https://mlsanity.com/paper/1803.03467/) (2018)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Learning Intents behind Interactions with Knowledge Graph for Recommendation](https://mlsanity.com/paper/2102.07057/) (2021)
- [Unifying Knowledge Graph Learning and Recommendation: Towards a Better Understanding of User Preferences](https://mlsanity.com/paper/1902.06236/) (2019)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [Translation-based Recommendation](https://mlsanity.com/paper/1707.02410/) (2017)
- [Analogical Inference for Multi-Relational Embeddings](https://mlsanity.com/paper/1705.02426/) (2017)
- [Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer](https://mlsanity.com/paper/2108.06625/) (2021)
- [Federated Social Recommendation with Graph Neural Network](https://mlsanity.com/paper/2111.10778/) (2021)
- [Augmenting Sequential Recommendation with Pseudo-Prior Items via Reversely Pre-training Transformer](https://mlsanity.com/paper/2105.00522/) (2021)
- [Reinforced Neighborhood Selection Guided Multi-Relational Graph Neural Networks](https://mlsanity.com/paper/2104.07886/) (2021)
- [CosRec: 2D Convolutional Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/1908.09972/) (2019)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [Relational Collaborative Filtering:Modeling Multiple Item Relations for Recommendation](https://mlsanity.com/paper/1904.12796/) (2019)
- [Modeling Sequences as Distributions with Uncertainty for Sequential Recommendation](https://mlsanity.com/paper/2106.06165/) (2021)
- [Exploiting Cross-Session Information for Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2107.00852/) (2021)
- [DSKReG: Differentiable Sampling on Knowledge Graph for Recommendation with Relational GNN](https://mlsanity.com/paper/2108.11883/) (2021)
- [Basket Recommendation with Multi-Intent Translation Graph Neural Network](https://mlsanity.com/paper/2010.11419/) (2020)

## Built on by (most cited first, 3 of 3)

- [Data Scarcity in Recommendation Systems: A Survey](https://mlsanity.com/paper/2312.10073/) (2023)
- [Episodes Discovery Recommendation with Multi-Source Augmentations](https://mlsanity.com/paper/2301.01737/) (2023)
- [Mutual Wasserstein Discrepancy Minimization for Sequential Recommendation](https://mlsanity.com/paper/2301.12197/) (2023)

## Abstract (arXiv)

Sequential Recommendation (SR) models user dynamics and predicts the next preferred items based on the user history. Existing SR methods model the 'was interacted before' item-item transitions observed in sequences, which can be viewed as an item relationship. However, there are multiple auxiliary item relationships, e.g., items from similar brands and with similar contents in real-world scenarios. Auxiliary item relationships describe item-item affinities in multiple different semantics and alleviate the long-lasting cold start problem in the recommendation. However, it remains a significant challenge to model auxiliary item relationships in SR. To simultaneously model high-order item-item transitions in sequences and auxiliary item relationships, we propose a Multi-relational Transformer capable of modeling auxiliary item relationships for SR (MT4SR). Specifically, we propose a novel self-attention module, which incorporates arbitrary item relationships and weights item relationships accordingly. Second, we regularize intra-sequence item relationships with a novel regularization module to supervise attentions computations. Third, for inter-sequence item relationship pairs, we introduce a novel inter-sequence related items modeling module. Finally, we conduct experiments on four benchmark datasets and demonstrate the effectiveness of MT4SR over state-of-the-art methods and the improvements on the cold start problem. The code is available at https://github.com/zfan20/MT4SR.