mlsanityData to 13 Oct 2026

Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images

arXiv 1810.06553 ↗First posted 14 Oct 2018cs.CVcited by 118

Not on the map. Why?

Abstract (arXiv)

In this paper, we introduce Recipe1M+, a new large-scale, structured corpus of over one million cooking recipes and 13 million food images. As the largest publicly available collection of recipe data, Recipe1M+ affords the ability to train high-capacity modelson aligned, multimodal data. Using these data, we train a neural network to learn a joint embedding of recipes and images that yields impressive results on an image-recipe retrieval task. Moreover, we demonstrate that regularization via the addition of a high-level classification objective both improves retrieval performance to rival that of humans and enables semantic vector arithmetic. We postulate that these embeddings will provide a basis for further exploration of the Recipe1M+ dataset and food and cooking in general. Code, data and models are publicly available.

Key facts. Cited by 118 papers in the map (data through 13 Oct 2026). First posted on arXiv 14 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 22; 2022: 18; 2023: 19; 2024: 24; 2025: 21; 2026: 14.

118 papers.

Sort:

By area:

  1. 2021Large Scale Visual Food Recognitioncited by 75
  2. 2022Real or Fake Text?: Investigating Human Ability to Detect Boundaries Between Human-Written and Machine-Generated Textcited by 63
  3. 2021Nutrition5k: Towards Automatic Nutritional Understanding of Generic Foodcited by 58
  4. 2021A Large-Scale Benchmark for Food Image Segmentationcited by 45
  5. 2021Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learningcited by 28
  6. 2021Pre-train or Annotate? Domain Adaptation with a Constrained Budgetcited by 27
  7. 2021Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptationcited by 20
  8. 2023FIRE: Food Image to REcipe generationcited by 18
  9. 2024MOPI-HFRS: A Multi-objective Personalized Health-aware Food Recommendation System with LLM-enhanced Interpretationcited by 18
  10. 2021Personalized Food Recommendation as Constrained Question Answering over a Large-scale Food Knowledge Graphcited by 17
  11. 2021Learning Structural Representations for Recipe Generation and Food Retrievalcited by 17
  12. 2022Learning Program Representations for Food Images and Cooking Recipescited by 16
38 more
  1. 2021SHARE: a System for Hierarchical Assistive Recipe Editingcited by 15
  2. 2021Seeing Through Clouds in Satellite Imagescited by 13
  3. 2022Recipe2Vec: Multi-modal Recipe Representation Learning with Graph Neural Networkscited by 12
  4. 2024RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Modelscited by 11
  5. 2021MultiSubs: A Large-scale Multimodal and Multilingual Datasetcited by 10
  6. 2021Continual learning in cross-modal retrievalcited by 10
  7. 2021Transferring Knowledge from Text to Video: Zero-Shot Anticipation for Procedural Actionscited by 10
  8. 2023Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Modelscited by 10
  9. 2023Dietary Assessment with Multimodal ChatGPT: A Systematic Analysiscited by 10
  10. 2024Mitigating Noisy Correspondence by Geometrical Structure Consistency Learningcited by 10
  11. 2024FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examinationcited by 10
  12. 2023From Cooking Recipes to Robot Task Trees -- Improving Planning Correctness and Task Efficiency by Leveraging LLMs with a Knowledge Networkcited by 9
  13. 2023GePSAn: Generative Procedure Step Anticipation in Cooking Videoscited by 9
  14. 2021Applications of knowledge graphs for food science and industrycited by 8
  15. 2021Learning Text-Image Joint Embedding for Efficient Cross-Modal Retrieval with Deep Feature Engineeringcited by 8
  16. 2023Learning to Substitute Ingredients in Recipescited by 8
  17. 2024Human Action Anticipation: A Surveycited by 8
  18. 2025SynthTRIPs: A Knowledge-Grounded Framework for Benchmark Query Generation for Personalized Tourism Recommenderscited by 8
  19. 2021CHEF: Cross-modal Hierarchical Embeddings for Food Domain Retrievalcited by 7
  20. 2021Egocentric Image Captioning for Privacy-Preserved Passive Dietary Intake Monitoringcited by 7
  21. 2022Task2Dial: A Novel Task and Dataset for Commonsense enhanced Task-based Dialogue Grounded in Documentscited by 7
  22. 2023Cultural Adaptation of Recipescited by 7
  23. 2024Unlocking Structure Measuring: Introducing PDD, an Automatic Metric for Positional Discourse Coherencecited by 7
  24. 2025A Survey on Large Language Models in Multimodal Recommender Systemscited by 7
  25. 2026Gemini Embedding 2: A Native Multimodal Embedding Model from Geminicited by 7
  26. 2022Transformer Decoders with MultiModal Regularization for Cross-Modal Food Retrievalcited by 6
  27. 2023SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detectioncited by 6
  28. 2024Consolidating Trees of Robotic Plans Generated Using Large Language Models to Improve Reliabilitycited by 6
  29. 2026GLEN-Bench: A Graph-Language based Benchmark for Nutritional Healthcited by 6
  30. 2022Plug-and-Play Recipe Generation with Content Planningcited by 5
  31. 2023The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagerycited by 5
  32. 2023Transferring Knowledge for Food Image Segmentation using Transformers and Convolutionscited by 5
  33. 2023Roll Up Your Sleeves: Working with a Collaborative and Engaging Task-Oriented Dialogue Systemcited by 5
  34. 2024OVFoodSeg: Elevating Open-Vocabulary Food Image Segmentation via Image-Informed Textual Representationcited by 5
  35. 2024Benchmark Granularity and Model Robustness for Image-Text Retrievalcited by 5
  36. 2024HANDI: Hand-Centric Text-and-Image Conditioned Video Generationcited by 5
  37. 2021Learning Joint Embedding with Modality Alignments for Cross-Modal Retrieval of Recipes and Food Imagescited by 4
  38. 2021FooDI-ML: a large multi-language dataset of food, drinks and groceries images and descriptionscited by 4

50 most cited of 118 All 118 (JSON)