# A review on vision-based analysis for automatic dietary assessment

arXiv: https://arxiv.org/abs/2108.02947 · HTML: https://mlsanity.com/paper/2108.02947/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 142 references that are in the map.
- First posted on arXiv 6 Aug 2021.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [Deep learning in agriculture: A survey](https://mlsanity.com/paper/1807.11809/) (2018)
- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [A Survey on Food Computing](https://mlsanity.com/paper/1808.07202/) (2018)
- [Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food](https://mlsanity.com/paper/2103.03375/) (2021)
- [ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network](https://mlsanity.com/paper/2008.05655/) (2020)
- [Visual Aware Hierarchy Based Food Recognition](https://mlsanity.com/paper/2012.03368/) (2020)
- [Multi-Task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation](https://mlsanity.com/paper/2004.13188/) (2020)
- [S2R-DepthNet: Learning a Generalizable Depth-specific Structural Representation](https://mlsanity.com/paper/2104.00877/) (2021)
- [Food Recommendation: Framework, Existing Solutions and Challenges](https://mlsanity.com/paper/1905.06269/) (2019)
- [An End-to-End Food Image Analysis System](https://mlsanity.com/paper/2102.00645/) (2021)
- [Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptation](https://mlsanity.com/paper/2103.07562/) (2021)
- [Two-view 3D Reconstruction for Food Volume Estimation](https://mlsanity.com/paper/1701.03330/) (2017)
- [Single-View Food Portion Estimation: Learning Image-to-Energy Mappings Using Generative Adversarial Networks](https://mlsanity.com/paper/1802.09670/) (2018)
- [Multi-Task Learning for Calorie Prediction on a Novel Large-Scale Recipe Dataset Enriched with Nutritional Information](https://mlsanity.com/paper/2011.01082/) (2020)
- [An Artificial Intelligence-Based System to Assess Nutrient Intake for Hospitalised Patients](https://mlsanity.com/paper/2003.08273/) (2020)
- [Partially Supervised Multi-Task Network for Single-View Dietary Assessment](https://mlsanity.com/paper/2008.00818/) (2020)
- [Self-Attention and Ingredient-Attention Based Model for Recipe Retrieval from Image Queries](https://mlsanity.com/paper/1911.01770/) (2019)
- [A Multi-Task Learning Approach for Meal Assessment](https://mlsanity.com/paper/1806.10343/) (2018)

## Built on by (most cited first, 9 of 9)

- [FoodSky: A Food-oriented Large Language Model that Passes the Chef and Dietetic Examination](https://mlsanity.com/paper/2406.10261/) (2024)
- [SeeDS: Semantic Separable Diffusion Synthesizer for Zero-shot Food Detection](https://mlsanity.com/paper/2310.04689/) (2023)
- [Synthesizing Knowledge-enhanced Features for Real-world Zero-shot Food Detection](https://mlsanity.com/paper/2402.09242/) (2024)
- [UMDFood: Vision-language models boost food composition compilation](https://mlsanity.com/paper/2306.01747/) (2023)
- [Feature-Suppressed Contrast for Self-Supervised Food Pre-training](https://mlsanity.com/paper/2308.03272/) (2023)
- [OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion](https://mlsanity.com/paper/2604.12356/) (2026)
- [How Much You Ate? Food Portion Estimation on Spoons](https://mlsanity.com/paper/2405.08717/) (2024)
- [A SAM based Tool for Semi-Automatic Food Annotation](https://mlsanity.com/paper/2410.19756/) (2024)
- [OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet](https://mlsanity.com/paper/2608.03428/) (2026)

## Abstract (arXiv)

Background: Maintaining a healthy diet is vital to avoid health-related issues, e.g., undernutrition, obesity and many non-communicable diseases. An indispensable part of the health diet is dietary assessment. Traditional manual recording methods are not only burdensome but time-consuming, and contain substantial biases and errors. Recent advances in Artificial Intelligence (AI), especially computer vision technologies, have made it possible to develop automatic dietary assessment solutions, which are more convenient, less time-consuming and even more accurate to monitor daily food intake. Scope and approach: This review presents Vision-Based Dietary Assessment (VBDA) architectures, including multi-stage architecture and end-to-end one. The multi-stage dietary assessment generally consists of three stages: food image analysis, volume estimation and nutrient derivation. The prosperity of deep learning makes VBDA gradually move to an end-to-end implementation, which applies food images to a single network to directly estimate the nutrition. The recently proposed end-to-end methods are also discussed. We further analyze existing dietary assessment datasets, indicating that one large-scale benchmark is urgently needed, and finally highlight critical challenges and future trends for VBDA. Key findings and conclusions: After thorough exploration, we find that multi-task end-to-end deep learning approaches are one important trend of VBDA. Despite considerable research progress, many challenges remain for VBDA due to the meal complexity. We also provide the latest ideas for future development of VBDA, e.g., fine-grained food analysis and accurate volume estimation. This review aims to encourage researchers to propose more practical solutions for VBDA.