# Attention-based sequential recommendation system using multimodal data

arXiv: https://arxiv.org/abs/2405.17959 · HTML: https://mlsanity.com/paper/2405.17959/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 28 May 2024.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashion](https://mlsanity.com/paper/1905.01866/) (2019)
- [Non-invasive Self-attention for Side Information Fusion in Sequential Recommendation](https://mlsanity.com/paper/2103.03578/) (2021)
- [Decoupled Side Information Fusion for Sequential Recommendation](https://mlsanity.com/paper/2204.11046/) (2022)
- [Recommender Systems with Heterogeneous Side Information](https://mlsanity.com/paper/1907.08679/) (2019)
- [A Multimodal Recommender System for Large-scale Assortment Generation in E-commerce](https://mlsanity.com/paper/1806.11226/) (2018)

## Built on by (most cited first, 1 of 1)

- [ORBIT -- Open Recommendation Benchmark for Reproducible Research with Hidden Tests](https://mlsanity.com/paper/2510.26095/) (2025)

## Abstract (arXiv)

Sequential recommendation systems that model dynamic preferences based on a use's past behavior are crucial to e-commerce. Recent studies on these systems have considered various types of information such as images and texts. However, multimodal data have not yet been utilized directly to recommend products to users. In this study, we propose an attention-based sequential recommendation method that employs multimodal data of items such as images, texts, and categories. First, we extract image and text features from pre-trained VGG and BERT and convert categories into multi-labeled forms. Subsequently, attention operations are performed independent of the item sequence and multimodal representations. Finally, the individual attention information is integrated through an attention fusion function. In addition, we apply multitask learning loss for each modality to improve the generalization performance. The experimental results obtained from the Amazon datasets show that the proposed method outperforms those of conventional sequential recommendation systems.