# Towards Bridging the Cross-modal Semantic Gap for Multi-modal Recommendation

arXiv: https://arxiv.org/abs/2407.05420 · HTML: https://mlsanity.com/paper/2407.05420/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Jul 2024.

## Built on by (most cited first, 2 of 2)

- [Structured Spectral Reasoning for Frequency-Adaptive Multimodal Recommendation](https://mlsanity.com/paper/2512.01372/) (2025)
- [Federated Vision-Language-Recommendation with Personalized Fusion](https://mlsanity.com/paper/2410.08478/) (2024)

## Abstract (arXiv)

Multi-modal recommendation greatly enhances the performance of recommender systems by modeling the auxiliary information from multi-modality contents. Most existing multi-modal recommendation models primarily exploit multimedia information propagation processes to enrich item representations and directly utilize modal-specific embedding vectors independently obtained from upstream pre-trained models. However, this might be inappropriate since the abundant task-specific semantics remain unexplored, and the cross-modality semantic gap hinders the recommendation performance. Inspired by the recent progress of the cross-modal alignment model CLIP, in this paper, we propose a novel CLIP Enhanced Recommender (CLIPER) framework to bridge the semantic gap between modalities and extract fine-grained multi-view semantic information. Specifically, we introduce a multi-view modality-alignment approach for representation extraction and measure the semantic similarity between modalities. Furthermore, we integrate the multi-view multimedia representations into downstream recommendation models. Extensive experiments conducted on three public datasets demonstrate the consistent superiority of our model over state-of-the-art multi-modal recommendation models.