A Survey on Large Language Models in Multimodal Recommender Systems
Not on the map. Why?
Abstract (arXiv)
Multimodal recommender systems (MRS) integrate heterogeneous user and item data, such as text, images, and structured information, to enhance recommendation performance. The emergence of large language models (LLMs) introduces new opportunities for MRS by enabling semantic reasoning, in-context learning, and dynamic input handling. Compared to earlier pre-trained language models (PLMs), LLMs offer greater flexibility and generalisation capabilities but also introduce challenges related to scalability and model accessibility. This survey presents a comprehensive review of recent work at the intersection of LLMs and MRS, focusing on prompting strategies, fine-tuning methods, and data adaptation techniques. We propose a novel taxonomy to characterise integration patterns, identify transferable techniques from related recommendation domains, provide an overview of evaluation metrics and datasets, and point to possible future directions. We aim to clarify the emerging role of LLMs in multimodal recommendation and support future research in this rapidly evolving field.
Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 77 of its 139 references that are in the map. First posted on arXiv 14 May 2025.
Builds on
77 of 139 references.
References · 77
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
- 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
- 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2017Deep reinforcement learning from human preferencescited by 3,912
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
- 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
- 2018Self-Attentive Sequential Recommendationcited by 1,184
- 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
38 more
- 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
- 2021P-Tuning v2: Prompt Tuning Can Be Comparable to Fine-tuning Universally Across Scales and Taskscited by 689
- 2024LLM2Vec: Large Language Models Are Secretly Powerful Text Encoderscited by 404
- 2022Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5)cited by 402
- 2017struc2vec: Learning Node Representations from Structural Identitycited by 215
- 2023Representation Learning with Large Language Models for Recommendationcited by 155
- 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
- 2021A Survey of Graph Neural Networks for Recommender Systems: Challenges, Methods, and Directionscited by 144
- 2023Where to Go Next for Recommender Systems? ID- vs. Modality-based Recommender Models Revisitedcited by 124
- 2018Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Imagescited by 118
- 2019Learning Disentangled Representations for Recommendationcited by 110
- 2021State Entropy Maximization with Random Encoders for Efficient Explorationcited by 108
- 2021Mining Latent Structures for Multimedia Recommendationcited by 79
- 2022Personalized Prompt Learning for Explainable Recommendationcited by 79
- 2019POG: Personalized Outfit Generation for Fashion Recommendation at Alibaba iFashioncited by 64
- 2021GRCN: Graph-Refined Convolutional Network for Multimedia Recommendation with Implicit Feedbackcited by 58
- 2025Knowledge Graph Retrieval-Augmented Generation for LLM-based Recommendationcited by 57
- 2022A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendationcited by 54
- 2023VIP5: Towards Multimodal Foundation Models for Recommendationcited by 46
- 2016Semantic Jitter: Dense Supervision for Visual Comparisons via Synthetic Imagescited by 45
- 2023Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systemscited by 45
- 2022Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendationcited by 42
- 2021Non-invasive Self-attention for Side Information Fusion in Sequential Recommendationcited by 39
- 2024KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extractioncited by 34
- 2024Harnessing Multimodal Large Language Models for Multimodal Sequential Recommendationcited by 33
- 2024Customizing Language Models with Instance-wise LoRA for Sequential Recommendationcited by 32
- 2022Personalized Showcases: Generating Multi-Modal Explanations for Recommendationscited by 30
- 2023IMF: Interactive Multimodal Fusion Model for Link Predictioncited by 28
- 2024Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recencycited by 28
- 2024Leveraging LLM Reasoning Enhances Personalized Recommender Systemscited by 28
- 2022CrossCBR: Cross-view Contrastive Learning for Bundle Recommendationcited by 26
- 2024Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyondcited by 25
- 2023Knowledge Graph Completion Models are Few-shot Learners: An Empirical Study of Relation Labeling in E-commerce with LLMscited by 24
- 2023Large Language Models for User Interest Journeyscited by 24
- 2024Enhancing Taobao Display Advertising with Multimodal Representations: Challenges, Approaches and Insightscited by 23
- 2022CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attentioncited by 22
- 2024LinRec: Linear Attention Mechanism for Long-term Sequential Recommender Systemscited by 21
- 2021Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendationcited by 18
50 most cited of 77 All 77 (JSON)
Built on by
Citing papers per year: 2025: 2; 2026: 5.
7 papers.
- 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Datacited by 21
- 2026RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignmentcited by 4
- 2026IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshucited by 2
- 2025From Raw Features to Effective Embeddings: A Three-Stage Approach for Multimodal Recipe Recommendationcited by 0
- 2026PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendationcited by 0
- 2026MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Searchcited by 0
- 2026MM-VeriRec: Failure-Guided Fusion for Verifiable Agentic Multimodal Recommendationcited by 0
- 2026MM-VeriRec: Failure-Guided Fusion for Verifiable Agentic Multimodal Recommendation21 Sep 2026
- 2026MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search13 Jul 2026
- 2026IDProxy: CTR Prediction with Multimodal LLMs for Cold-Start Recommendation at Xiaohongshu2 Mar 2026
- 2026RecGOAT: Graph Optimal Adaptive Transport for LLM-Enhanced Multimodal Recommendation with Dual Semantic Alignment31 Jan 2026
- 2026PRISM: Purified Representation and Integrated Semantic Modeling for Generative Sequential Recommendation23 Jan 2026
- 2025From Raw Features to Effective Embeddings: A Three-Stage Approach for Multimodal Recipe Recommendation24 Nov 2025
- 2025Uni-MoE-2.0-Omni: Scaling Language-Centric Omnimodal Large Model with Advanced MoE, Training and Data16 Nov 2025