mlsanityData to 14 Oct 2026

CulturalMenuBench: Probing the Knowledge-Application Gap in Multimodal Culinary Reasoning

arXiv 2609.03526 ↗First posted 3 Sep 2026cs.AIcited by 0

On the map: LLM reasoning → Multimodal reasoning

also in 1 more
Abstract (arXiv)

Multimodal language models achieve near-ceiling scores on food recognition benchmarks, yet it remains unclear whether this success reflects genuine cultural understanding or mere visual matching. To probe this distinction, we introduce CulturalMenuBench, a benchmark of 4,870 items in 10 languages across 18 regions; its 10 tasks pair final-dish and step-by-step cooking images with ingredients, procedural text, and regional labels, spanning basic recognition to process-grounded cultural attribution. Evaluating 12 models exposes a substantial knowledge-application gap: models exceeding 94% on standard multiple-choice tasks drop to at most 56% when attributing dishes to Chinese regional cuisines, despite an identical four-way format. Diagnostic analyses explain why: error patterns are consistent with random guessing, accuracy tracks visual distinctiveness rather than cultural structure, and models classify cuisines more accurately from dish names alone than from images (+7-18 points). The knowledge is thus present but cannot be activated through visual input. An ablation confirms these tasks genuinely require procedural evidence: removing sequential cooking images selectively degrades process-grounded tasks while others remain stable. Overall, CulturalMenuBench shows that near-perfect recognition can conceal an inability to apply cultural knowledge, motivating training that explicitly connects perception, procedure, and cultural context. Code and data are publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 10 of its 38 references that are in the map. First posted on arXiv 3 Sep 2026.

Builds on

10 of 38 references.

References · 10

  1. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  2. 2024DeepSeek-V3 Technical Reportcited by 811
  3. 2026Kimi K2.5: Visual Agentic Intelligencecited by 501
  4. 2026Gemma 4 Technical Reportcited by 259
  5. 2024Massively Multi-Cultural Knowledge Acquisition & LM Benchmarkingcited by 63
  6. 2024CulturalTeaming: AI-Assisted Interactive Red-Teaming for Challenging LLMs' (Lack of) Multicultural Knowledgecited by 28
  7. 2025MultiNRC: A Challenging and Native Multilingual Reasoning Evaluation Benchmark for LLMscited by 16
  8. 2024Exploring Visual Culture Awareness in GPT-4V: A Comprehensive Probingcited by 12
  9. 2026IndustryBench: Probing the Industrial Knowledge Boundaries of LLMscited by 4
  10. 2026IndustryBench-MIPU: Benchmarking Multi-Image Attribute Value Extraction for Industrial Productscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture
  2. 2024FoodMLLM-JP: Leveraging Multimodal Large Language Models for Japanese Recipe Generation
  3. 2025World in a Frame: Understanding Culture Mixing as a New Challenge for Vision-Language Models
  4. 2025When Tom Eats Kimchi: Evaluating Cultural Bias of Multimodal Large Language Models in Cultural Mixture Contexts
  5. 2024FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination
  6. 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisines
  7. 2025Seeing Culture: A Benchmark for Visual Reasoning and Grounding
  8. 2023Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models