mlsanityData to 7 Oct 2026

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

arXiv 2606.04986 ↗First posted 3 Jun 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 25 of its 36 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

25 of 36 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Corvid: Improving Multimodal Large Language Models Towards Chain-of-Thought Reasoning10 Jul
  2. 2024CaLoRAify: Calorie Estimation with Visual-Text Pairing and LoRA-Driven Visual Language Models13 Dec
  3. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Step15 Nov
  4. 2024MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine11 Jul

Other references · 21

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 653
  6. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  7. 2023EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thoughtcited by 276
  8. 2025ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Drivingcited by 190
  9. 2018Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Imagescited by 118
  10. 2025AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoningcited by 108
  11. 2021Large Scale Visual Food Recognitioncited by 75
  12. 2021Nutrition5k: Towards Automatic Nutritional Understanding of Generic Foodcited by 58
9 more
  1. 2023FoodLMM: A Versatile Food Assistant using Large Multi-modal Modelcited by 32
  2. 2025Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasonercited by 24
  3. 2023FIRE: Food Image to REcipe generationcited by 18
  4. 2016Simultaneous Food Localization and Recognitioncited by 14
  5. 2024LLaVA-Chef: A Multi-modal Generative Model for Food Recipescited by 14
  6. 2024RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Modelscited by 11
  7. 2024From Canteen Food to Daily Meals: Generalizing Food Recognition to More Practical Scenarioscited by 9
  8. 2024Retrieval Augmented Recipe Generationcited by 3
  9. 2025MM-Food-100K: A 100,000-Sample Multimodal Food Intelligence Dataset with Verifiable Provenancecited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain
  2. 2026OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice
  3. 2025Are Vision-Language Models Ready for Dietary Assessment? Exploring the Next Frontier in AI-Powered Food Image Recognition
  4. 2026FoodSense: A Multisensory Food Dataset and Benchmark for Predicting Taste, Smell, Texture, and Sound from Images
  5. 2026FAM-Bench: A Multimodal Benchmark for Condition-Aware Food-as-Medicine Reasoning
  6. 2025January Food Benchmark (JFB): A Public Benchmark Dataset and Evaluation Suite for Multimodal Food Analysis
  7. 2022Vision and Structured-Language Pretraining for Cross-Modal Food Retrieval
  8. 2023UMDFood: Vision-language models boost food composition compilation