# Adaptive Multi-Modalities Fusion in Sequential Recommendation Systems

arXiv: https://arxiv.org/abs/2308.15980 · HTML: https://mlsanity.com/paper/2308.15980/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Aug 2023.

## Built on by (most cited first, 14 of 14)

- [Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2408.09698/) (2024)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [FineRec:Exploring Fine-grained Sequential Recommendation](https://mlsanity.com/paper/2404.12975/) (2024)
- [Disentangling ID and Modality Effects for Session-based Recommendation](https://mlsanity.com/paper/2404.12969/) (2024)
- [Modality-Balanced Learning for Multimedia Recommendation](https://mlsanity.com/paper/2408.06360/) (2024)
- [Diffusion-based Multi-modal Synergy Interest Network for Click-through Rate Prediction](https://mlsanity.com/paper/2508.21460/) (2025)
- [MRSE: An Efficient Multi-modality Retrieval System for Large Scale E-commerce](https://mlsanity.com/paper/2408.14968/) (2024)
- [Automatic Feature Fairness in Recommendation via Adversaries](https://mlsanity.com/paper/2309.15418/) (2023)
- [MMSRARec: Summarization and Retrieval Augumented Sequential Recommendation Based on Multimodal Large Language Model](https://mlsanity.com/paper/2512.20916/) (2025)
- [An item is worth one token in Multimodal Large Language Models-based Sequential Recommendation](https://mlsanity.com/paper/2511.05885/) (2025)
- [MealRec: Multi-granularity Sequential Modeling via Hierarchical Diffusion Models for Micro-Video Recommendation](https://mlsanity.com/paper/2603.01926/) (2026)
- [CAMMSR: Category-Guided Attentive Mixture of Experts for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.04320/) (2026)
- [VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2603.17450/) (2026)
- [OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2610.16038/) (2026)

## Abstract (arXiv)

In sequential recommendation, multi-modal information (e.g., text or image) can provide a more comprehensive view of an item's profile. The optimal stage (early or late) to fuse modality features into item representations is still debated. We propose a graph-based approach (named MMSR) to fuse modality features in an adaptive order, enabling each modality to prioritize either its inherent sequential nature or its interplay with other modalities. MMSR represents each user's history as a graph, where the modality features of each item in a user's history sequence are denoted by cross-linked nodes. The edges between homogeneous nodes represent intra-modality sequential relationships, and the ones between heterogeneous nodes represent inter-modality interdependence relationships. During graph propagation, MMSR incorporates dual attention, differentiating homogeneous and heterogeneous neighbors. To adaptively assign nodes with distinct fusion orders, MMSR allows each node's representation to be asynchronously updated through an update gate. In scenarios where modalities exhibit stronger sequential relationships, the update gate prioritizes updates among homogeneous nodes. Conversely, when the interdependent relationships between modalities are more pronounced, the update gate prioritizes updates among heterogeneous nodes. Consequently, MMSR establishes a fusion order that spans a spectrum from early to late modality fusion. In experiments across six datasets, MMSR consistently outperforms state-of-the-art models, and our graph propagation methods surpass other graph neural networks. Additionally, MMSR naturally manages missing modalities.