# Multimodal Pretraining, Adaptation, and Generation for Recommendation: A Survey

arXiv: https://arxiv.org/abs/2404.00621 · HTML: https://mlsanity.com/paper/2404.00621/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Mar 2024.

## Built on by (most cited first, 25 of 25)

- [Towards Next-Generation LLM-based Recommender Systems: A Survey and Beyond](https://mlsanity.com/paper/2410.19744/) (2024)
- [DecAlign: Hierarchical Cross-Modal Alignment for Decoupled Multimodal Representation Learning](https://mlsanity.com/paper/2503.11892/) (2025)
- [Understanding Layer Significance in LLM Alignment](https://mlsanity.com/paper/2410.17875/) (2024)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [Mitigating Visual Knowledge Forgetting in MLLM Instruction-tuning via Modality-decoupled Gradient Descent](https://mlsanity.com/paper/2502.11740/) (2025)
- [Efficient and Effective Adaptation of Multimodal Foundation Models in Sequential Recommendation](https://mlsanity.com/paper/2411.02992/) (2024)
- [Structured Spectral Reasoning for Frequency-Adaptive Multimodal Recommendation](https://mlsanity.com/paper/2512.01372/) (2025)
- [TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddings](https://mlsanity.com/paper/2603.04772/) (2026)
- [MOON2.0: Dynamic Modality-balanced Multimodal Representation Learning for E-commerce Product Understanding](https://mlsanity.com/paper/2511.12449/) (2025)
- [Diffusion Models in Recommendation Systems: A Survey](https://mlsanity.com/paper/2501.10548/) (2025)
- [COVID-19 on YouTube: A Data-Driven Analysis of Sentiment, Toxicity, and Content Recommendations](https://mlsanity.com/paper/2412.17180/) (2024)
- [Tree of Preferences for Diversified Recommendation](https://mlsanity.com/paper/2601.02386/) (2025)
- [Federated Vision-Language-Recommendation with Personalized Fusion](https://mlsanity.com/paper/2410.08478/) (2024)
- [MISS: Multi-Modal Tree Indexing and Searching with Lifelong Sequential Behavior for Retrieval Recommendation](https://mlsanity.com/paper/2508.14515/) (2025)
- [Personalized Image Aesthetic Assessment via Preference-rich Sample Mining and Cohort Merging](https://mlsanity.com/paper/2607.15752/) (2026)
- [ST-DPGAN: A Privacy-preserving Framework for Spatiotemporal Data Generation](https://mlsanity.com/paper/2406.03404/) (2024)
- [MAJL: A Model-Agnostic Joint Learning Framework for Music Source Separation and Pitch Estimation](https://mlsanity.com/paper/2501.03689/) (2025)
- [MDE: Modality Discrimination Enhancement for Multi-modal Recommendation](https://mlsanity.com/paper/2502.18481/) (2025)
- [Intrinsic and Extrinsic Factor Disentanglement for Recommendation in Various Context Scenarios](https://mlsanity.com/paper/2503.03524/) (2025)
- [Cross-Modal Attention Network with Dual Graph Learning in Multimodal Recommendation](https://mlsanity.com/paper/2601.11151/) (2026)
- [VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models](https://mlsanity.com/paper/2603.12625/) (2026)
- [MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search](https://mlsanity.com/paper/2607.11030/) (2026)
- [PaletteID: Prototype-Composed Semantic Identifiers for Multimodal CTR Prediction](https://mlsanity.com/paper/2607.29000/) (2026)
- [RecPFN: Prior-Fitted Networks for In-Context-Based Recommendations](https://mlsanity.com/paper/2608.19735/) (2026)
- [OT-FlowRec: Optimal Transport-Guided Multi-Bridge Flow for Multimodal Sequential Recommendation](https://mlsanity.com/paper/2610.16038/) (2026)

## Abstract (arXiv)

Personalized recommendation serves as a ubiquitous channel for users to discover information tailored to their interests. However, traditional recommendation models primarily rely on unique IDs and categorical features for user-item matching, potentially overlooking the nuanced essence of raw item contents across multiple modalities such as text, image, audio, and video. This underutilization of multimodal data poses a limitation to recommender systems, especially in multimedia services like news, music, and short-video platforms. The recent advancements in large multimodal models offer new opportunities and challenges in developing content-aware recommender systems. This survey seeks to provide a comprehensive exploration of the latest advancements and future trajectories in multimodal pretraining, adaptation, and generation techniques, as well as their applications in enhancing recommender systems. Furthermore, we discuss current open challenges and opportunities for future research in this dynamic domain. We believe that this survey, alongside the curated resources, will provide valuable insights to inspire further advancements in this evolving landscape.