mlsanityData to 7 Oct 2026

NutriBench-Kitchen: Benchmarking Embodied AI for Nutrition Management

arXiv 2609.07135 ↗First posted 7 Sep 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

An embodied kitchen assistant must do more than recognize food in isolated frames. It must track ingredient states over time and integrate visual observations with recipe and nutritional knowledge to support constraint-aware decision-making. We formalize this capability as Embodied Nutrition Management: perceiving nutrition-relevant events, maintaining a persistent food state, and using it for knowledge-grounded planning. Existing benchmarks evaluate static food understanding or embodied cooking actions, but do not measure whether an agent can continuously update and use nutrition-relevant states in dynamic kitchens. To fill this gap, we introduce NutriBench-Kitchen, a benchmark containing 1,500 manually verified question--answer pairs from 160 cooking videos. It covers five task families: Ingredient Entry, Memory Management, Recipe Query, Long-Term Planning, and Short-Term Planning, spanning food-state construction, maintenance, knowledge retrieval, and decision-making across different planning horizons. Evaluations of proprietary and open-source large vision-language models reveal a substantial gap from human performance, particularly in quantitative ingredient estimation, long-term state tracking, and reasoning under interacting constraints. We further introduce Nutri-Vgent, a diagnostic long-video agent with separate episodic, food-state, and recipe memories. Its consistent improvements demonstrate the value of explicit state representations and structured memory for nutrition management. Together, NutriBench-Kitchen and Nutri-Vgent provide a testbed for studying persistent state tracking and knowledge-grounded reasoning in dynamic kitchens. Code is available at https://github.com/V1ol1n/NutriBench-Kitchen.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 20 of its 28 references that are in the map. First posted on arXiv 7 Sep 2026.

Builds on

20 of 28 references.

References · 20

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  2. 2021Ego4D: Around the World in 3,000 Hours of Egocentric Videocited by 1,530
  3. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  4. 2018Scaling Egocentric Vision: The EPIC-KITCHENS Datasetcited by 789
  5. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
  6. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learningcited by 439
  7. 2020The EPIC-KITCHENS Dataset: Collection, Challenges and Baselinescited by 221
  8. 2018Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Imagescited by 118
  9. 2025HD-EPIC: A Highly-Detailed Egocentric Video Datasetcited by 101
  10. 2021Nutrition5k: Towards Automatic Nutritional Understanding of Generic Foodcited by 58
  11. 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisinescited by 39
  12. 2025Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spacescited by 33
8 more
  1. 2025Vgent: Graph-based Retrieval-Reasoning-Augmented Generation For Long Video Understandingcited by 30
  2. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culturecited by 25
  3. 2024ET-Plan-Bench: Embodied Task-level Planning Benchmark Towards Spatial-Temporal Cognition with Foundation Modelscited by 11
  4. 2023Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Modelscited by 10
  5. 2025CookBench: A Long-Horizon Embodied Planning Benchmark for Complex Cooking Scenarioscited by 10
  6. 2025Advancing Food Nutrition Estimation via Visual-Ingredient Feature Fusioncited by 5
  7. 2025Comprehensive Evaluation of Large Multimodal Models for Nutrition Analysis: A New Benchmark Enriched with Contextual Metadatacited by 3
  8. 2025SFOOD: A Multimodal Benchmark for Comprehensive Food Attribute Analysis Beyond RGB with Spectral Insightscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Nutri-ATLAS: Embodied Agent for Tabulated Lookup and Assistance for Smarter nutrition
  2. 2026DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain
  3. 2026FAM-Bench: A Multimodal Benchmark for Condition-Aware Food-as-Medicine Reasoning
  4. 2026ReGraph: Learning to Generate Recipe Graphs from Food Images
  5. 2026OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice
  6. 2023FoodLMM: A Versatile Food Assistant using Large Multi-modal Model
  7. 2026Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning
  8. 2024PizzaCommonSense: Learning to Model Commonsense Reasoning about Intermediate Steps in Cooking Recipes