Thought-For-Food: Reasoning Chain Induced Food Visual Question Answering
On the map: LLM reasoning → Multimodal reasoning
Abstract (arXiv)
The immense diversity in the culture and culinary of Indian cuisines calls attention to the major shortcoming of the existing Visual Question Answering(VQA) systems which are inclined towards the foods from Western region. Recent attempt towards building a VQA dataset for Indian food is a step towards addressing this challenge. However, their approach towards VQA follows a two-step process in which the answer is generated first, followed by the explanation of the expected answer. In this work, we claim that food VQA requires to follow a multi-step reasoning process to arrive at an accurate answer, especially in the context of India food, which involves understanding complex culinary context and identifying relationships between various food items. With this hypothesis we create reasoning chains upon the QA with minimal human intervention. We fine-tune smaller LLMs and VLMs with auto-validated reasoning chains and further train them using reinforcement learning with larger data. With augmentation of reasoning chains, we observed accuracy improvement of an average 10 percentage points on the baseline. We provide detailed analysis in terms the effect of addition of reasoning chains for the Indian Food VQA task. Index Terms - FoodVQA, Reasoning Chains, Reinforcement Learning, Knowledge Graph.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal reasoning, one of 17 directions in LLM reasoning. Builds on 19 of its 33 references that are in the map. First posted on arXiv 3 Nov 2025.
Builds on
19 of 33 references.
Closest ancestors · 2
Same direction, newest first
- 2024Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning25 Mar
- 2023Boosting the Power of Small Multimodal Reasoning Models to Match Larger Models with Self-Consistency Training23 Nov
Other references · 17
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2015VQA: Visual Question Answeringcited by 2,489
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimizationcited by 296
- 2024DPO Meets PPO: Reinforced Token Optimization for RLHFcited by 126
- 2024Low-Rank Few-Shot Adaptation of Vision-Language Modelscited by 103
- 2019GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answeringcited by 77
5 more
- 2018Inverse Cooking: Recipe Generation from Food Imagescited by 47
- 2024WorldCuisines: A Massive-Scale Benchmark for Multilingual and Multicultural Visual Question Answering on Global Cuisinescited by 39
- 2023FoodLMM: A Versatile Food Assistant using Large Multi-modal Modelcited by 32
- 2025Debiasing Multimodal Large Language Models via Noise-Aware Preference Optimizationcited by 14
- 2025IADGPT: Unified LVLM for Few-Shot Industrial Anomaly Detection, Localization, and Reasoning via In-Context Learningcited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain
- 2026Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning
- 2022Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
- 2025MultiFoodhat: A potential new paradigm for intelligent food quality inspection
- 2022Co-VQA : Answering by Interactive Sub Question Sequence
- 2025Enhancing Food-Domain Question Answering with a Multimodal Knowledge Graph: Hybrid QA Generation and Diversity Analysis
- 2024FoodieQA: A Multimodal Dataset for Fine-Grained Understanding of Chinese Food Culture
- 2024FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination