# Multi-Modal Self-Supervised Learning for Recommendation

arXiv: https://arxiv.org/abs/2302.10632 · HTML: https://mlsanity.com/paper/2302.10632/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2023.

## Built on by (most cited first, 30 of 45)

- [UniTime: A Language-Empowered Unified Model for Cross-Domain Time Series Forecasting](https://mlsanity.com/paper/2310.09751/) (2023)
- [DiffKG: Knowledge Graph Diffusion Model for Recommendation](https://mlsanity.com/paper/2312.16890/) (2023)
- [MISSRec: Pre-training and Transferring Multi-modal Interest-aware Sequence Representation for Recommendation](https://mlsanity.com/paper/2308.11175/) (2023)
- [IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT](https://mlsanity.com/paper/2404.02059/) (2024)
- [Towards Unified Multi-Modal Personalization: Large Vision-Language Models for Generative Recommendation and Beyond](https://mlsanity.com/paper/2403.10667/) (2024)
- [NativE: Multi-modal Knowledge Graph Completion in the Wild](https://mlsanity.com/paper/2406.17605/) (2024)
- [Knowledge Graph Self-Supervised Rationalization for Recommendation](https://mlsanity.com/paper/2307.02759/) (2023)
- [DiffMM: Multi-Modal Diffusion Model for Recommendation](https://mlsanity.com/paper/2406.11781/) (2024)
- [Embedding in Recommender Systems: A Survey](https://mlsanity.com/paper/2310.18608/) (2023)
- [A Survey on Self-Supervised Learning for Non-Sequential Tabular Data](https://mlsanity.com/paper/2402.01204/) (2024)
- [GPT-ST: Generative Pre-Training of Spatio-Temporal Graph Neural Networks](https://mlsanity.com/paper/2311.04245/) (2023)
- [Multimodality Invariant Learning for Multimedia-Based New Item Recommendation](https://mlsanity.com/paper/2405.15783/) (2024)
- [SelfGNN: Self-Supervised Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/2405.20878/) (2024)
- [ID-Free Not Risk-Free: LLM-Powered Agents Unveil Risks in ID-Free Recommender Systems](https://mlsanity.com/paper/2409.11690/) (2024)
- [LD4MRec: Simplifying and Powering Diffusion Model for Multimedia Recommendation](https://mlsanity.com/paper/2309.15363/) (2023)
- [A Comprehensive Survey on Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2404.03354/) (2024)
- [Structured Spectral Reasoning for Frequency-Adaptive Multimodal Recommendation](https://mlsanity.com/paper/2512.01372/) (2025)
- [Modality-Balanced Learning for Multimedia Recommendation](https://mlsanity.com/paper/2408.06360/) (2024)
- [Transferable and Forecastable User Targeting Foundation Model](https://mlsanity.com/paper/2412.12468/) (2024)
- [Disentangling and Generating Modalities for Recommendation in Missing Modality Scenarios](https://mlsanity.com/paper/2504.16352/) (2025)
- [Pareto Invariant Representation Learning for Multimedia Recommendation](https://mlsanity.com/paper/2308.04706/) (2023)
- [How Expressive are Graph Neural Networks in Recommendation?](https://mlsanity.com/paper/2308.11127/) (2023)
- [Mixed Supervised Graph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2404.15954/) (2024)
- [Do We Really Need Graph Convolution During Training? Light Post-Training Graph-ODE for Efficient Recommendation](https://mlsanity.com/paper/2407.18910/) (2024)
- [RAG-VisualRec: An Open Resource for Vision- and Text-Enhanced Retrieval-Augmented Generation in Recommendation](https://mlsanity.com/paper/2506.20817/) (2025)
- [RoleMAG: Learning Neighbor Roles in Multimodal Graphs](https://mlsanity.com/paper/2604.12271/) (2026)
- [UGMAE: A Unified Framework for Graph Masked Autoencoders](https://mlsanity.com/paper/2402.08023/) (2024)
- [Integrating Biological and Machine Intelligence: Attention Mechanisms in Brain-Computer Interfaces](https://mlsanity.com/paper/2502.19281/) (2025)
- [MMHCL: Multi-Modal Hypergraph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2504.16576/) (2025)
- [SVTime: Small Time Series Forecasting Models Informed by "Physics" of Large Vision Model Forecasters](https://mlsanity.com/paper/2510.09780/) (2025)

## Abstract (arXiv)

The online emergence of multi-modal sharing platforms (eg, TikTok, Youtube) is powering personalized recommender systems to incorporate various modalities (eg, visual, textual and acoustic) into the latent user representations. While existing works on multi-modal recommendation exploit multimedia content features in enhancing item embeddings, their model representation capability is limited by heavy label reliance and weak robustness on sparse user behavior data. Inspired by the recent progress of self-supervised learning in alleviating label scarcity issue, we explore deriving self-supervision signals with effectively learning of modality-aware user preference and cross-modal dependencies. To this end, we propose a new Multi-Modal Self-Supervised Learning (MMSSL) method which tackles two key challenges. Specifically, to characterize the inter-dependency between the user-item collaborative view and item multi-modal semantic view, we design a modality-aware interactive structure learning paradigm via adversarial perturbations for data augmentation. In addition, to capture the effects that user's modality-aware interaction pattern would interweave with each other, a cross-modal contrastive learning approach is introduced to jointly preserve the inter-modal semantic commonality and user preference diversity. Experiments on real-world datasets verify the superiority of our method in offering great potential for multimedia recommendation over various state-of-the-art baselines. The implementation is released at: https://github.com/HKUDS/MMSSL.