# FoodCHA: Multi-Modal LLM Agent for Fine-Grained Food Analysis

arXiv: https://arxiv.org/abs/2605.05499 · HTML: https://mlsanity.com/paper/2605.05499/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 12 of its 48 references that are in the map.
- First posted on arXiv 6 May 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 12 of 12)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [Conversational Health Agents: A Personalized LLM-Powered Agent Framework](https://mlsanity.com/paper/2310.02374/) (2023)
- [Adapting Large Language Models to Domains via Reading Comprehension](https://mlsanity.com/paper/2309.09530/) (2023)
- [FoodLMM: A Versatile Food Assistant using Large Multi-modal Model](https://mlsanity.com/paper/2312.14991/) (2023)
- [Res-VMamba: Fine-Grained Food Category Visual Classification Using Selective State Space Models with Deep Residual Learning](https://mlsanity.com/paper/2402.15761/) (2024)
- [On Domain-Adaptive Post-Training for Multimodal Large Language Models](https://mlsanity.com/paper/2411.19930/) (2024)
- [Food-500 Cap: A Fine-Grained Food Caption Benchmark for Evaluating Vision-Language Models](https://mlsanity.com/paper/2308.03151/) (2023)
- [VL-SAM-V2: Open-World Object Detection with General and Specific Query Fusion](https://mlsanity.com/paper/2505.18986/) (2025)
- [An Explorative Analysis of SVM Classifier and ResNet50 Architecture on African Food Classification](https://mlsanity.com/paper/2505.13923/) (2025)
- [A Closed-Loop Multi-Agent System Driven by LLMs for Meal-Level Personalized Nutrition Management](https://mlsanity.com/paper/2601.04491/) (2026)
- [Improving Food Image Recognition with Noisy Vision Transformer](https://mlsanity.com/paper/2503.18997/) (2025)

## Abstract (arXiv)

The widespread adoption of camera-equipped mobile devices and wearables has enabled convenient capture of meal images, making food recognition a key component for real time dietary monitoring. However, real-world food images present challenges due to high intra-class similarity and the frequent presence of multiple food items within a single image. While deep learning models achieve strong performance in coarse grained classification, they often struggle to capture fine-grained attributes such as cooking style. Moreover, open-ended generation in modern vision-language models can produce non-canonical labels, limiting their practical deployment. We propose FoodCHA, a multimodal agentic framework that reformulates food recognition as a hierarchical decision-making process. By progressively anchoring predictions, FoodCHA guides subcategory identification using high-level categories and guides cooking style recognition using subcategories, improving semantic consistency and attribute-level discrimination. To ensure practical deployability, FoodCHA utilizes the compact Moondream-2B vision language model, which provides strong reasoning capability while maintaining lower computational and memory overhead. Experiments on FoodNExTDB show that FoodCHA outperforms Food-Llama-3.2-11B by 13.8% and 38.2% in category and subcategory recognition precision, respectively, and achieves a striking 153.2% improvement in cooking style classification precision.