mlsanityData to 14 Oct 2026

MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation

arXiv 2308.11175 ↗First posted 22 Aug 2023cs.IR cs.AI cs.MMcited by 31

Not on the map. Why?

Abstract (arXiv)

The goal of sequential recommendation (SR) is to predict a user's potential interested items based on her/his historical interaction sequences. Most existing sequential recommenders are developed based on ID features, which, despite their widespread use, often underperform with sparse IDs and struggle with the cold-start problem. Besides, inconsistent ID mappings hinder the model's transferability, isolating similar recommendation domains that could have been co-optimized. This paper aims to address these issues by exploring the potential of multi-modal information in learning robust and generalizable sequence representations. We propose MISSRec, a multi-modal pre-training and transfer learning framework for SR. On the user side, we design a Transformer-based encoder-decoder model, where the contextual encoder learns to capture the sequence-level multi-modal user interests while a novel interest-aware decoder is developed to grasp item-modality-interest relations for better sequence representation. On the candidate item side, we adopt a dynamic fusion module to produce user-adaptive item representation, providing more precise matching between users and items. We pre-train the model with contrastive learning objectives and fine-tune it in an efficient manner. Extensive experiments demonstrate the effectiveness and flexibility of MISSRec, promising a practical solution for real-world recommendation scenarios. Data and code are available on urlhttps://github.com/gimpong/MM23-MISSRec.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 54 of its 97 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

54 of 97 references.

References · 54

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  6. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  7. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  8. 2023A Survey of Large Language Modelscited by 2,576
  9. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  11. 2018Self-Attentive Sequential Recommendationcited by 1,184
  12. 2020LightGCN: Simplifying and Powering Graph Convolution Network for Recommendationcited by 1,139
38 more
  1. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  2. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  3. 2021Towards a Unified View of Parameter-Efficient Transfer Learningcited by 872
  4. 2015Session-based Recommendations with Recurrent Neural Networkscited by 695
  5. 2017Deep Interest Network for Click-Through Rate Predictioncited by 544
  6. 2018Personalized Top-N Sequential Recommendation via Convolutional Sequence Embeddingcited by 476
  7. 2020S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximizationcited by 311
  8. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 242
  9. 2016Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendationcited by 152
  10. 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
  11. 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
  12. 2021Contrastive Learning for Representation Degeneration Problem in Sequential Recommendationcited by 121
  13. 2022Towards Universal Sequence Representation Learning for Recommender Systemscited by 121
  14. 2018A Simple Convolutional Generative Network for Next Item Recommendationcited by 120
  15. 2021Causal Intervention for Leveraging Popularity Bias in Recommendationcited by 116
  16. 2019Sequential Recommender Systems: Challenges, Progress and Prospectscited by 115
  17. 2019Multi-Interest Network with Dynamic Routing for Recommendation at Tmallcited by 114
  18. 2022Learning Vector-Quantized Item Representation for Transferable Sequential Recommenderscited by 112
  19. 2019Hierarchical Gating Networks for Sequential Recommendationcited by 101
  20. 2022Filter-enhanced MLP is All You Need for Sequential Recommendationcited by 96
  21. 2020Controllable Multi-Interest Framework for Recommendationcited by 93
  22. 2022Intent Contrastive Learning for Sequential Recommendationcited by 91
  23. 2021Mining Latent Structures for Multimedia Recommendationcited by 79
  24. 2021Cross-Domain Recommendation: Challenges, Progress, and Prospectscited by 74
  25. 2021SimpleX: A Simple and Strong Baseline for Collaborative Filteringcited by 65
  26. 2018Sequence-Aware Recommender Systemscited by 58
  27. 2021GRCN: Graph-Refined Convolutional Network for Multimedia Recommendation with Implicit Feedbackcited by 58
  28. 2022Bootstrap Latent Representations for Multi-modal Recommendationcited by 55
  29. 2023Multi-Modal Self-Supervised Learning for Recommendationcited by 45
  30. 2020The Connection Between Popularity Bias, Calibration, and Fairness in Recommendationcited by 35
  31. 2022TransRec: Learning Transferable Recommendation from Mixture-of-Modality Feedbackcited by 27
  32. 2018Perceive Your Users in Depth: Learning Universal User Representations from Multiple E-commerce Taskscited by 24
  33. 2020Detecting Beneficial Feature Interactions for Recommender Systemscited by 22
  34. 2022Re4: Learning to Re-contrast, Re-attend, Re-construct for Multi-interest Recommendationcited by 19
  35. 2023A Comprehensive Survey on Multimodal Recommender Systems: Taxonomy, Evaluation, and Future Directionscited by 17
  36. 2021UPRec: User-Aware Pre-training for Recommender Systemscited by 16
  37. 2021Click-Through Rate Prediction with Multi-Modal Hypergraphscited by 12
  38. 2021Why Do We Click: Visual Impression-aware News Recommendationcited by 10

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 14; 2026: 9.

31 papers.

Sort:

By area:

  1. 2024IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFTcited by 26
  2. 2024LEARN: Knowledge Adaptation from Large Language Model to Recommendation for Practical Industrial Applicationcited by 21
  3. 2024PMG : Personalized Multimodal Generation with Large Language Modelscited by 19
  4. 2025Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendationcited by 15
  5. 2025Multi-Modal Multi-Behavior Sequential Recommendation with Conditional Diffusion-Based Feature Denoisingcited by 14
  6. 2025Multimodal Quantitative Language for Generative Recommendationcited by 12
  7. 2025Universal Item Tokenization for Transferable Generative Recommendationcited by 7
  8. 2024Semantic Codebook Learning for Dynamic Recommendation Modelscited by 6
  9. 2024Captions Speak Louder than Images: Generalizing Foundation Models for E-commerce from High-quality Multimodal Instruction Datacited by 6
  10. 2025EvdCLIP: Improving Vision-Language Retrieval with Entity Visual Descriptions from Large Language Modelscited by 6
  11. 2025MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understandingcited by 5
  12. 2025MIM: Multi-modal Content Interest Modeling Paradigm for User Behavior Modelingcited by 3
19 more
  1. 2025Multi-Aspect Cross-modal Quantization for Generative Recommendationcited by 3
  2. 2024BiVRec: Bidirectional View-based Multimodal Sequential Recommendationcited by 1
  3. 2024RAT: Retrieval-Augmented Transformer for Click-Through Rate Predictioncited by 1
  4. 2024Federated Vision-Language-Recommendation with Personalized Fusioncited by 1
  5. 2025Catalog-Native LLM: Speaking Item-ID Dialect with Less Entanglement for Recommendationcited by 1
  6. 2026Learning Variable-Length Tokenization for Generative Recommendationcited by 1
  7. 2026SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendationcited by 1
  8. 2025Bridging Domain Gaps between Pretrained Multimodal Models and Recommendationscited by 0
  9. 2025Intrinsic and Extrinsic Factor Disentanglement for Recommendation in Various Context Scenarioscited by 0
  10. 2025A Novel Mamba-based Sequential Recommendation Methodcited by 0
  11. 2025Large-scale User Game Lifecycle Representation Learningcited by 0
  12. 2025Q-BERT4Rec: Quantized Semantic-ID Representation Learning for Multimodal Recommendationcited by 0
  13. 2026CAMMSR: Category-Guided Attentive Mixture of Experts for Multimodal Sequential Recommendationcited by 0
  14. 2026VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendationcited by 0
  15. 2026Aligning Multimodal Sequential Recommendations via Robust Direct Preference Optimization with Sparse MoEcited by 0
  16. 2026TME-PSR: Time-aware, Multi-interest, and Explanation Personalization for Sequential Recommendationcited by 0
  17. 2026Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendationcited by 0
  18. 2026Cross-Country Code-Mixing for Generative Recommendationcited by 0
  19. 2026OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendationcited by 0