# Latent Structure Mining with Contrastive Modality Fusion for Multimedia Recommendation

arXiv: https://arxiv.org/abs/2111.00678 · HTML: https://mlsanity.com/paper/2111.00678/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 86 references that are in the map.
- First posted on arXiv 1 Nov 2021.

## Builds on (most cited first, 30 of 53)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Graph Attention Networks](https://mlsanity.com/paper/1710.10903/) (2017)
- [Simplifying Graph Convolutional Networks](https://mlsanity.com/paper/1902.07153/) (2019)
- [Contrastive Multiview Coding](https://mlsanity.com/paper/1906.05849/) (2019)
- [Graph Contrastive Learning with Augmentations](https://mlsanity.com/paper/2010.13902/) (2020)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [Deep Graph Infomax](https://mlsanity.com/paper/1809.10341/) (2018)
- [Multimodal Machine Learning: A Survey and Taxonomy](https://mlsanity.com/paper/1705.09406/) (2017)
- [Link Prediction Based on Graph Neural Networks](https://mlsanity.com/paper/1802.09691/) (2018)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Learning Representations by Maximizing Mutual Information Across Views](https://mlsanity.com/paper/1906.00910/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Deep Graph Contrastive Representation Learning](https://mlsanity.com/paper/2006.04131/) (2020)
- [Ups and Downs: Modeling the Visual Evolution of Fashion Trends with One-Class Collaborative Filtering](https://mlsanity.com/paper/1602.01585/) (2016)
- [Graph Contrastive Learning with Adaptive Augmentation](https://mlsanity.com/paper/2010.14945/) (2020)
- [Graph Structure Learning for Robust Graph Neural Networks](https://mlsanity.com/paper/2005.10203/) (2020)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [S^3-Rec: Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization](https://mlsanity.com/paper/2008.07873/) (2020)
- [Self-supervised Graph Learning for Recommendation](https://mlsanity.com/paper/2010.10783/) (2020)
- [Self-Supervised Learning by Cross-Modal Audio-Video Clustering](https://mlsanity.com/paper/1911.12667/) (2019)
- [MISA: Modality-Invariant and -Specific Representations for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2005.03545/) (2020)
- [Learning Discrete Structures for Graph Neural Networks](https://mlsanity.com/paper/1903.11960/) (2019)
- [Iterative Deep Graph Learning for Graph Neural Networks: Better and Robust Node Embeddings](https://mlsanity.com/paper/2006.13009/) (2020)
- [Audio-Visual Instance Discrimination with Cross-Modal Agreement](https://mlsanity.com/paper/2004.12943/) (2020)

## Built on by (most cited first, 18 of 18)

- [Multimodal Recommender Systems: A Survey](https://mlsanity.com/paper/2302.03883/) (2023)
- [Stealthy Attack on Large Language Model based Recommendation](https://mlsanity.com/paper/2402.14836/) (2024)
- [Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models](https://mlsanity.com/paper/2402.08670/) (2024)
- [Multimodality Invariant Learning for Multimedia-Based New Item Recommendation](https://mlsanity.com/paper/2405.15783/) (2024)
- [LD4MRec: Simplifying and Powering Diffusion Model for Multimedia Recommendation](https://mlsanity.com/paper/2309.15363/) (2023)
- [GUME: Graphs and User Modalities Enhancement for Long-Tail Multimodal Recommendation](https://mlsanity.com/paper/2407.12338/) (2024)
- [RAGAR: Retrieval Augmented Personalized Image Generation Guided by Recommendation](https://mlsanity.com/paper/2505.01657/) (2025)
- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [A Comprehensive Survey on Self-Supervised Learning for Recommendation](https://mlsanity.com/paper/2404.03354/) (2024)
- [Mining Stable Preferences: Adaptive Modality Decorrelation for Multimedia Recommendation](https://mlsanity.com/paper/2306.14179/) (2023)
- [Multi-Modal Recommendation Unlearning for Legal, Licensing, and Modality Constraints](https://mlsanity.com/paper/2405.15328/) (2024)
- [Diffusion Models in Recommendation Systems: A Survey](https://mlsanity.com/paper/2501.10548/) (2025)
- [Multi-Aspect Cross-modal Quantization for Generative Recommendation](https://mlsanity.com/paper/2511.15122/) (2025)
- [Semantic Item Graph Enhancement for Multimodal Recommendation](https://mlsanity.com/paper/2508.06154/) (2025)
- [Hi-SAM: A Hierarchical Structure-Aware Multi-modal Framework for Large-Scale Recommendation](https://mlsanity.com/paper/2602.11799/) (2026)
- [Dreaming User Multimodal Representation Guided by The Platonic Representation Hypothesis for Micro-Video Recommendation](https://mlsanity.com/paper/2410.03538/) (2024)
- [EGRA:Toward Enhanced Behavior Graphs and Representation Alignment for Multimodal Recommendation](https://mlsanity.com/paper/2508.16170/) (2025)
- [DReX: An Explainable Deep Learning-based Multimodal Recommendation Framework](https://mlsanity.com/paper/2602.19702/) (2026)

## Abstract (arXiv)

Recent years have witnessed growing interests in multimedia recommendation, which aims to predict whether a user will interact with an item with multimodal contents. Previous studies focus on modeling user-item interactions with multimodal features included as side information. However, this scheme is not well-designed for multimedia recommendation. Firstly, only collaborative item-item relationships are implicitly modeled through high-order item-user-item co-occurrences. We argue that the latent semantic item-item structures underlying these multimodal contents could be beneficial for learning better item representations and assist the recommender models to comprehensively discover candidate items. Secondly, previous studies disregard the fine-grained multimodal fusion. Although having access to multiple modalities might allow us to capture rich information, we argue that the simple coarse-grained fusion by linear combination or concatenation in previous work is insufficient to fully understand content information and item relationships.To this end, we propose a latent structure MIning with ContRastive mOdality fusion method (MICRO for brevity). To be specific, we devise a novel modality-aware structure learning module, which learns item-item relationships for each modality. Based on the learned modality-aware latent item relationships, we perform graph convolutions that explicitly inject item affinities to modality-aware item representations. Then, we design a novel contrastive method to fuse multimodal features. These enriched item representations can be plugged into existing collaborative filtering methods to make more accurate recommendations. Extensive experiments on real-world datasets demonstrate the superiority of our method over state-of-the-art baselines.