mlsanityData to 7 Oct 2026

Visual Recipe Flow: A Dataset for Learning Visual State Changes of Objects with Recipe Flows

arXiv 2209.05840 ↗First posted 13 Sep 2022cs.CL cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

We present a new multimodal dataset called Visual Recipe Flow, which enables us to learn each cooking action result in a recipe text. The dataset consists of object state changes and the workflow of the recipe text. The state change is represented as an image pair, while the workflow is represented as a recipe flow graph (r-FG). The image pairs are grounded in the r-FG, which provides the cross-modal relation. With our dataset, one can try a range of applications, from multimodal commonsense reasoning and procedural text generation.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 13 of its 28 references that are in the map. First posted on arXiv 13 Sep 2022.

Builds on

13 of 28 references.

References · 13

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  5. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 946
  6. 2018Learning a Text-Video Embedding from Incomplete and Heterogeneous Datacited by 103
  7. 2018RecipeQA: A Challenge Dataset for Multimodal Comprehension of Cooking Recipescited by 74
  8. 2018Tracking State Changes in Procedural Text: A Challenge Dataset and Models for Process Paragraph Comprehensioncited by 62
  9. 2017Simulating Action Dynamics with Neural Process Networkscited by 53
  10. 2020A Dataset for Tracking Entities in Open Domain Procedural Textcited by 37
  11. 2022Learning Program Representations for Food Images and Cooking Recipescited by 16
  12. 2020Multi-modal Cooking Workflow Construction for Food Recipescited by 15
1 more
  1. 2019CITE: A Corpus of Image-Text Discourse Relationscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 3; 2025: 3; 2026: 1.

8 papers.

  1. 2024SCHEMA: State CHangEs MAtter for Procedure Planning in Instructional Videoscited by 24
  2. 2024J-CRe3: A Japanese Conversation Dataset for Real-world Reference Resolutioncited by 6
  3. 2024COM Kitchens: An Unedited Overhead-view Video Dataset as a Vision-Language Benchmarkcited by 3
  4. 2025STATUS Bench: A Rigorous Benchmark for Evaluating Object State Understanding in Vision-Language Modelscited by 3
  5. 2025MET-Bench: Multimodal Entity Tracking for Evaluating the Limitations of Vision-Language and Reasoning Modelscited by 1
  6. 2023Towards Flow Graph Prediction of Open-Domain Procedural Textscited by 0
  7. 2025Towards Multimodal Graph Large Language Modelcited by 0
  8. 2026ReGraph: Learning to Generate Recipe Graphs from Food Imagescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025CookAnything: A Framework for Flexible and Consistent Multi-Step Recipe Image Generation
  2. 2025Chain-of-Cooking:Cooking Process Visualization via Bidirectional Chain-of-Thought Guidance
  3. 2022Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
  4. 2026Enhancing Action and Ingredient Modeling for Semantically Grounded Recipe Generation
  5. 2024PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipes
  6. 2025CookingDiffusion: Cooking Procedural Image Generation with Stable Diffusion
  7. 2026MMSkills: Towards Multimodal Skills for General Visual Agents
  8. 2026How Does India Cook Biryani?