# Predefined domain specific embeddings of food concepts and recipes: A case study on heterogeneous recipe datasets

arXiv: https://arxiv.org/abs/2302.01005 · HTML: https://mlsanity.com/paper/2302.01005/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 44 references that are in the map.
- First posted on arXiv 2 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Multi-layer Representation Learning for Medical Concepts](https://mlsanity.com/paper/1602.05568/) (2016)
- [Learning Cross-Modal Embeddings with Adversarial Networks for Cooking Recipes and Food Images](https://mlsanity.com/paper/1905.01273/) (2019)
- [Clinical Concept Embeddings Learned from Massive Sources of Multimodal Medical Data](https://mlsanity.com/paper/1804.01486/) (2018)
- [Cross-Modal Retrieval in the Cooking Context: Learning Semantic Text-Image Embeddings](https://mlsanity.com/paper/1804.11146/) (2018)
- [Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learning](https://mlsanity.com/paper/2103.13061/) (2021)
- [Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networks](https://mlsanity.com/paper/2205.12396/) (2022)

## Abstract (arXiv)

Although recipe data are very easy to come by nowadays, it is really hard to find a complete recipe dataset - with a list of ingredients, nutrient values per ingredient, and per recipe, allergens, etc. Recipe datasets are usually collected from social media websites where users post and publish recipes. Usually written with little to no structure, using both standardized and non-standardized units of measurement. We collect six different recipe datasets, publicly available, in different formats, and some including data in different languages. Bringing all of these datasets to the needed format for applying a machine learning (ML) pipeline for nutrient prediction [1], [2], includes data normalization using dictionary-based named entity recognition (NER), rule-based NER, as well as conversions using external domain-specific resources. From the list of ingredients, domain-specific embeddings are created using the same embedding space for all recipes - one ingredient dataset is generated. The result from this normalization process is two corpora - one with predefined ingredient embeddings and one with predefined recipe embeddings. On all six recipe datasets, the ML pipeline is evaluated. The results from this use case also confirm that the embeddings merged using the domain heuristic yield better results than the baselines.