# From Raw Features to Effective Embeddings: A Three-Stage Approach for Multimodal Recipe Recommendation

arXiv: https://arxiv.org/abs/2511.19176 · HTML: https://mlsanity.com/paper/2511.19176/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 24 Nov 2025.

## Builds on (most cited first, 10 of 10)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation](https://mlsanity.com/paper/2002.02126/) (2020)
- [Bootstrap Latent Representations for Multi-modal Recommendation](https://mlsanity.com/paper/2207.05969/) (2022)
- [A Tale of Two Graphs: Freezing and Denoising Graph Structures for Multimodal Recommendation](https://mlsanity.com/paper/2211.06924/) (2022)
- [MOPI-HFRS: A Multi-objective Personalized Health-aware Food Recommendation System with LLM-enhanced Interpretation](https://mlsanity.com/paper/2412.08847/) (2024)
- [Enhancing Dyadic Relations with Homogeneous Graphs for Multimodal Recommendation](https://mlsanity.com/paper/2301.12097/) (2023)
- [MEGCF: Multimodal Entity Graph Collaborative Filtering for Personalized Recommendation](https://mlsanity.com/paper/2210.07443/) (2022)
- [A Survey on Large Language Models in Multimodal Recommender Systems](https://mlsanity.com/paper/2505.09777/) (2025)
- [MMHCL: Multi-Modal Hypergraph Contrastive Learning for Recommendation](https://mlsanity.com/paper/2504.16576/) (2025)
- [From "What to Eat?" to Perfect Recipe: ChefMind's Chain-of-Exploration for Ambiguous User Intent in Recipe Recommendation](https://mlsanity.com/paper/2509.18226/) (2025)

## Abstract (arXiv)

Recipe recommendation has become an essential task in web-based food platforms. A central challenge is effectively leveraging rich multimodal features beyond user-recipe interactions. Our analysis shows that even simple uses of multimodal signals yield competitive performance, suggesting that systematic enhancement of these signals is highly promising. We propose TESMR, a 3-stage framework for recipe recommendation that progressively refines raw multimodal features into effective embeddings through: (1) content-based enhancement using foundation models with multimodal comprehension, (2) relation-based enhancement via message propagation over user-recipe interactions, and (3) learning-based enhancement through contrastive learning with learnable embeddings. Experiments on two real-world datasets show that TESMR outperforms existing methods, achieving 7-15% higher Recall@10.