mlsanityData to 7 Oct 2026

RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipes

arXiv 1809.00812 ↗First posted 4 Sep 2018cs.CL cs.CVcited by 74

Not on the map. Why?

Abstract (arXiv)

Understanding and reasoning about cooking recipes is a fruitful research direction towards enabling machines to interpret procedural text. In this work, we introduce RecipeQA, a dataset for multimodal comprehension of cooking recipes. It comprises of approximately 20K instructional recipes with multiple modalities such as titles, descriptions and aligned set of images. With over 36K automatically generated question-answer pairs, we design a set of comprehension and reasoning tasks that require joint understanding of images and text, capturing the temporal flow of events and making sense of procedural knowledge. Our preliminary results indicate that RecipeQA will serve as a challenging test bed and an ideal benchmark for evaluating machine comprehension systems. The data and leaderboard are available at http://hucvl.github.io/recipeqa.

Key facts. Cited by 74 papers in the map (data through 7 Oct 2026). First posted on arXiv 4 Sep 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 12; 2023: 13; 2024: 17; 2025: 8; 2026: 7.

74 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2022MedMCQA : A Large-scale Multi-Subject Multi-Choice Dataset for Medical domain Question Answeringcited by 525
  3. 2021InfographicVQAcited by 430
  4. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  5. 2021WebQA: Multihop and Multimodal QAcited by 109
  6. 2021WebSRC: A Dataset for Web-Based Structural Reading Comprehensioncited by 98
  7. 2023Fine-tuning Multimodal LLMs to Follow Zero-shot Demonstrative Instructionscited by 76
  8. 2024MMIU: Multimodal Multi-image Understanding for Evaluating Large Vision-Language Modelscited by 61
  9. 2022MIntRec: A New Dataset for Multimodal Intent Recognitioncited by 41
  10. 2021N24News: A New Dataset for Multimodal News Classificationcited by 40
  11. 2023Multimodal Procedural Planning via Dual Text-Image Promptingcited by 40
  12. 2021Goal-Oriented Script Constructioncited by 30
38 more
  1. 2021Revamping Cross-Modal Recipe Retrieval with Hierarchical Transformers and Self-supervised Learningcited by 28
  2. 2022HybriDialogue: An Information-Seeking Dialogue Dataset Grounded on Tabular and Textual Datacited by 28
  3. 2021English Machine Reading Comprehension Datasets: A Surveycited by 26
  4. 2024M3SciQA: A Multi-Modal Multi-Document Scientific QA Benchmark for Evaluating Foundation Modelscited by 26
  5. 2024Distractor Generation in Multiple-Choice Tasks: A Survey of Methods, Datasets, and Evaluationcited by 25
  6. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  7. 2022Modern Question Answering Datasets and Benchmarks: A Surveycited by 23
  8. 2021Time-Stamped Language Model: Teaching Language Models to Understand the Flow of Eventscited by 22
  9. 2021ICDAR 2021 Competition on Document VisualQuestion Answeringcited by 19
  10. 2021Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language Modelscited by 18
  11. 2021Understanding Multimodal Procedural Knowledge by Sequencing Multimodal Instructional Manualscited by 18
  12. 2022Process Knowledge-Infused AI: Towards User-level Explainability, Interpretability, and Safetycited by 18
  13. 2025MRAMG-Bench: A Comprehensive Benchmark for Advancing Multimodal Retrieval-Augmented Multimodal Generationcited by 15
  14. 2022Reasoning about Actions over Visual and Linguistic Modalities: A Surveycited by 10
  15. 2023Lexi: Self-Supervised Learning of the UI Languagecited by 10
  16. 2024JDocQA: Japanese Document Question Answering Dataset for Generative Language Modelscited by 9
  17. 2021Applications of knowledge graphs for food science and industrycited by 8
  18. 2022Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flowscited by 8
  19. 2026MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoningcited by 8
  20. 2022"Mama Always Had a Way of Explaining Things So I Could Understand'': A Dialogue Corpus for Learning to Construct Explanationscited by 6
  21. 2026GLEN-Bench: A Graph-Language based Benchmark for Nutritional Healthcited by 6
  22. 2021Cross-Modal Coherence for Text-to-Image Retrievalcited by 5
  23. 2023The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagerycited by 5
  24. 2022RecipeMind: Guiding Ingredient Choices from Food Pairing to Recipe Completion using Cascaded Set Transformercited by 4
  25. 2023The Role of Semantic Parsing in Understanding Procedural Textcited by 4
  26. 2025Long-horizon Visual Instruction Generation with Logic and Attribute Self-reflectioncited by 4
  27. 2025Breaking Language Barriers in Visual Language Models via Multilingual Textual Regularizationcited by 4
  28. 2026Beyond Human Annotation: Recent Advances in Data Generation Methods for Document Intelligencecited by 4
  29. 2023KTRL+F: Knowledge-Augmented In-Document Searchcited by 3
  30. 2023BloomVQA: Assessing Hierarchical Multi-modal Comprehensioncited by 3
  31. 2024Graph Guided Question Answer Generation for Procedural Question-Answeringcited by 3
  32. 2026DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domaincited by 3
  33. 2021Latent Alignment of Procedural Concepts in Multimodal Recipescited by 2
  34. 2021Towards Solving Multimodal Comprehensioncited by 2
  35. 2022A Rich Recipe Representation as Plan to Support Expressive Multi Modal Queries on Recipe Content and Preparation Processcited by 2
  36. 2022Modeling Temporal-Modal Entity Graph for Procedural Multimodal Machine Comprehensioncited by 2
  37. 2023Assorted, Archetypal and Annotated Two Million (3A2M) Cooking Recipes Dataset based on Active Learningcited by 2
  38. 2023Machine Reading Comprehension using Case-based Reasoningcited by 2

50 most cited of 74 All 74 (JSON)