# MCEN: Bridging Cross-Modal Gap between Cooking Recipes and Dish Images with Latent Variable Model

arXiv: https://arxiv.org/abs/2004.01095 · HTML: https://mlsanity.com/paper/2004.01095/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Apr 2020.

## Built on by (most cited first, 16 of 16)

- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning](https://mlsanity.com/paper/2103.13061/) (2021)
- [Learning Program Representations for Food Images and Cooking Recipes](https://mlsanity.com/paper/2203.16071/) (2022)
- [Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature Engineering](https://mlsanity.com/paper/2110.11592/) (2021)
- [Learning TFIDF Enhanced Joint Embedding for Recipe-Image Cross-Modal Retrieval Service](https://mlsanity.com/paper/2108.00724/) (2021)
- [Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrieval](https://mlsanity.com/paper/2204.09730/) (2022)
- [Paired Cross-Modal Data Augmentation for Fine-Grained Image-to-Text Retrieval](https://mlsanity.com/paper/2207.14428/) (2022)
- [The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagery](https://mlsanity.com/paper/2304.02016/) (2023)
- [Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Images](https://mlsanity.com/paper/2108.03788/) (2021)
- [Cross-lingual Adaptation for Recipe Retrieval with Mixup](https://mlsanity.com/paper/2205.03891/) (2022)
- [Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval](https://mlsanity.com/paper/2212.04267/) (2022)
- [Transformer-based Cross-Modal Recipe Embeddings with Large Batch Training](https://mlsanity.com/paper/2205.04948/) (2022)
- [MALM: Mask Augmentation based Local Matching for Food-Recipe Retrieval](https://mlsanity.com/paper/2305.11327/) (2023)
- [Efficient Deep Feature Calibration for Cross-Modal Joint Embedding Learning](https://mlsanity.com/paper/2108.00705/) (2021)
- [Towards Unbiased Cross-Modal Representation Learning for Food Image-to-Recipe Retrieval](https://mlsanity.com/paper/2511.15201/) (2025)
- [SIMMER: Cross-Modal Food Image--Recipe Retrieval via MLLM-Based Embedding](https://mlsanity.com/paper/2604.15628/) (2026)

## Abstract (arXiv)

Nowadays, driven by the increasing concern on diet and health, food computing has attracted enormous attention from both industry and research community. One of the most popular research topics in this domain is Food Retrieval, due to its profound influence on health-oriented applications. In this paper, we focus on the task of cross-modal retrieval between food images and cooking recipes. We present Modality-Consistent Embedding Network (MCEN) that learns modality-invariant representations by projecting images and texts to the same embedding space. To capture the latent alignments between modalities, we incorporate stochastic latent variables to explicitly exploit the interactions between textual and visual features. Importantly, our method learns the cross-modal alignments during training but computes embeddings of different modalities independently at inference time for the sake of efficiency. Extensive experimental results clearly demonstrate that the proposed MCEN outperforms all existing approaches on the benchmark Recipe1M dataset and requires less computational cost.