# Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

arXiv: https://arxiv.org/abs/2606.04986 · HTML: https://mlsanity.com/paper/2606.04986/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 25 of 25)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models](https://mlsanity.com/paper/2503.06749/) (2025)
- [LLaVA-CoT: Let Vision Language Models Reason Step-by-Step](https://mlsanity.com/paper/2411.10440/) (2024)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought](https://mlsanity.com/paper/2305.15021/) (2023)
- [ReCogDrive: A Reinforced Cognitive Framework for End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.08052/) (2025)
- [Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images](https://mlsanity.com/paper/1810.06553/) (2018)
- [MAVIS: Mathematical Visual Instruction Tuning with an Automatic Data Engine](https://mlsanity.com/paper/2407.08739/) (2024)
- [AlphaDrive: Unleashing the Power of VLMs in Autonomous Driving via Reinforcement Learning and Reasoning](https://mlsanity.com/paper/2503.07608/) (2025)
- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food](https://mlsanity.com/paper/2103.03375/) (2021)
- [FoodLMM: A Versatile Food Assistant using Large Multi-modal Model](https://mlsanity.com/paper/2312.14991/) (2023)
- [Chart-R1: Chain-of-Thought Supervision and Reinforcement for Advanced Chart Reasoner](https://mlsanity.com/paper/2507.15509/) (2025)
- [FIRE: Food Image to REcipe generation](https://mlsanity.com/paper/2308.14391/) (2023)
- [Simultaneous Food Localization and Recognition](https://mlsanity.com/paper/1604.07953/) (2016)
- [LLaVA-Chef: A Multi-modal Generative Model for Food Recipes](https://mlsanity.com/paper/2408.16889/) (2024)
- [RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models](https://mlsanity.com/paper/2407.12730/) (2024)
- [Corvid: Improving Multimodal Large Language Models Towards Chain-of-Thought Reasoning](https://mlsanity.com/paper/2507.07424/) (2025)
- [From Canteen Food to Daily Meals: Generalizing Food Recognition to More Practical Scenarios](https://mlsanity.com/paper/2403.07403/) (2024)
- [CaLoRAify: Calorie Estimation with Visual-Text Pairing and LoRA-Driven Visual Language Models](https://mlsanity.com/paper/2412.09936/) (2024)
- [Retrieval Augmented Recipe Generation](https://mlsanity.com/paper/2411.08715/) (2024)
- [MM-Food-100K: A 100,000-Sample Multimodal Food Intelligence Dataset with Verifiable Provenance](https://mlsanity.com/paper/2508.10429/) (2025)

## Abstract (arXiv)

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.