# An End-to-end Food Portion Estimation Framework Based on Shape Reconstruction from Monocular Image

arXiv: https://arxiv.org/abs/2308.01810 · HTML: https://mlsanity.com/paper/2308.01810/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 26 references that are in the map.
- First posted on arXiv 3 Aug 2023.

## Builds on (most cited first, 14 of 14)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food](https://mlsanity.com/paper/2103.03375/) (2021)
- [Unsupervised Monocular Depth and Ego-motion Learning with Structure and Semantics](https://mlsanity.com/paper/1906.05717/) (2019)
- [Online Continual Learning For Visual Food Classification](https://mlsanity.com/paper/2108.06781/) (2021)
- [Visual Aware Hierarchy Based Food Recognition](https://mlsanity.com/paper/2012.03368/) (2020)
- [Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptation](https://mlsanity.com/paper/2103.07562/) (2021)
- [Single-View Food Portion Estimation: Learning Image-to-Energy Mappings Using Generative Adversarial Networks](https://mlsanity.com/paper/1802.09670/) (2018)
- [Improving Dietary Assessment Via Integrated Hierarchy Food Classification](https://mlsanity.com/paper/2109.02736/) (2021)
- [Image Based Food Energy Estimation With Depth Domain Adaptation](https://mlsanity.com/paper/2208.12153/) (2022)

## Built on by (most cited first, 15 of 15)

- [Food Portion Estimation via 3D Object Scaling](https://mlsanity.com/paper/2404.12257/) (2024)
- [MetaFood CVPR 2024 Challenge on Physically Informed 3D Food Reconstruction: Methods and Results](https://mlsanity.com/paper/2407.09285/) (2024)
- [MFP3D: Monocular Food Portion Estimation Leveraging 3D Point Clouds](https://mlsanity.com/paper/2411.10492/) (2024)
- [An Improved Encoder-Decoder Framework for Food Energy Estimation](https://mlsanity.com/paper/2309.00468/) (2023)
- [Diffusion Model with Clustering-based Conditioning for Food Image Generation](https://mlsanity.com/paper/2309.00199/) (2023)
- [Muti-Stage Hierarchical Food Classification](https://mlsanity.com/paper/2309.01075/) (2023)
- [Automatic Recognition of Food Ingestion Environment from the AIM-2 Wearable Sensor](https://mlsanity.com/paper/2405.07827/) (2024)
- [Comprehensive Evaluation of Large Multimodal Models for Nutrition Analysis: A New Benchmark Enriched with Contextual Metadata](https://mlsanity.com/paper/2507.07048/) (2025)
- [Food Portion Estimation: From Pixels to Calories](https://mlsanity.com/paper/2602.05078/) (2026)
- [OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion](https://mlsanity.com/paper/2604.12356/) (2026)
- [How Much You Ate? Food Portion Estimation on Spoons](https://mlsanity.com/paper/2405.08717/) (2024)
- [Shape-Preserving Generation of Food Images for Automatic Dietary Assessment](https://mlsanity.com/paper/2408.13358/) (2024)
- [PortionNet: Distilling 3D Geometric Knowledge for Food Nutrition Estimation](https://mlsanity.com/paper/2512.22304/) (2025)
- [DietDelta: A Vision-Language Approach for Dietary Assessment via Before-and-After Images](https://mlsanity.com/paper/2604.06352/) (2026)
- [V-Nutri: Dish-Level Nutrition Estimation from Egocentric Cooking Videos](https://mlsanity.com/paper/2604.11913/) (2026)

## Abstract (arXiv)

Dietary assessment is a key contributor to monitoring health status. Existing self-report methods are tedious and time-consuming with substantial biases and errors. Image-based food portion estimation aims to estimate food energy values directly from food images, showing great potential for automated dietary assessment solutions. Existing image-based methods either use a single-view image or incorporate multi-view images and depth information to estimate the food energy, which either has limited performance or creates user burdens. In this paper, we propose an end-to-end deep learning framework for food energy estimation from a monocular image through 3D shape reconstruction. We leverage a generative model to reconstruct the voxel representation of the food object from the input image to recover the missing 3D information. Our method is evaluated on a publicly available food image dataset Nutrition5k, resulting a Mean Absolute Error (MAE) of 40.05 kCal and Mean Absolute Percentage Error (MAPE) of 11.47% for food energy estimation. Our method uses RGB image as the only input at the inference stage and achieves competitive results compared to the existing method requiring both RGB and depth information.