# Food Portion Estimation via 3D Object Scaling

arXiv: https://arxiv.org/abs/2404.12257 · HTML: https://mlsanity.com/paper/2404.12257/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 41 references that are in the map.
- First posted on arXiv 18 Apr 2024.

## Builds on (most cited first, 17 of 17)

- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depth](https://mlsanity.com/paper/2302.12288/) (2023)
- [OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation](https://mlsanity.com/paper/2301.07525/) (2023)
- [Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food](https://mlsanity.com/paper/2103.03375/) (2021)
- [Visual Aware Hierarchy Based Food Recognition](https://mlsanity.com/paper/2012.03368/) (2020)
- [Multi-Task Image-Based Dietary Assessment for Food Recognition and Portion Size Estimation](https://mlsanity.com/paper/2004.13188/) (2020)
- [An End-to-End Food Image Analysis System](https://mlsanity.com/paper/2102.00645/) (2021)
- [Towards Learning Food Portion From Monocular Images With Cross-Domain Feature Adaptation](https://mlsanity.com/paper/2103.07562/) (2021)
- [Two-view 3D Reconstruction for Food Volume Estimation](https://mlsanity.com/paper/1701.03330/) (2017)
- [An Integrated System for Mobile Image-Based Dietary Assessment](https://mlsanity.com/paper/2110.01754/) (2021)
- [Long-tailed Food Classification](https://mlsanity.com/paper/2210.14748/) (2022)
- [Improving Dietary Assessment Via Integrated Hierarchy Food Classification](https://mlsanity.com/paper/2109.02736/) (2021)
- [An End-to-end Food Portion Estimation Framework Based on Shape Reconstruction from Monocular Image](https://mlsanity.com/paper/2308.01810/) (2023)
- [Image Based Food Energy Estimation With Depth Domain Adaptation](https://mlsanity.com/paper/2208.12153/) (2022)
- [DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion](https://mlsanity.com/paper/2310.11702/) (2023)
- [An Improved Encoder-Decoder Framework for Food Energy Estimation](https://mlsanity.com/paper/2309.00468/) (2023)

## Built on by (most cited first, 12 of 12)

- [MetaFood CVPR 2024 Challenge on Physically Informed 3D Food Reconstruction: Methods and Results](https://mlsanity.com/paper/2407.09285/) (2024)
- [MFP3D: Monocular Food Portion Estimation Leveraging 3D Point Clouds](https://mlsanity.com/paper/2411.10492/) (2024)
- [Comprehensive Evaluation of Large Multimodal Models for Nutrition Analysis: A New Benchmark Enriched with Contextual Metadata](https://mlsanity.com/paper/2507.07048/) (2025)
- [Food Portion Estimation: From Pixels to Calories](https://mlsanity.com/paper/2602.05078/) (2026)
- [Implicit-Scale 3D Reconstruction for Multi-Food Volume Estimation from Monocular Images](https://mlsanity.com/paper/2602.13041/) (2026)
- [Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception](https://mlsanity.com/paper/2604.09886/) (2026)
- [PortionNet: Distilling 3D Geometric Knowledge for Food Nutrition Estimation](https://mlsanity.com/paper/2512.22304/) (2025)
- [DietDelta: A Vision-Language Approach for Dietary Assessment via Before-and-After Images](https://mlsanity.com/paper/2604.06352/) (2026)
- [V-Nutri: Dish-Level Nutrition Estimation from Egocentric Cooking Videos](https://mlsanity.com/paper/2604.11913/) (2026)
- [Lightweight Neural Framework for Robust 3D Volume and Surface Estimation from Multi-View Images](https://mlsanity.com/paper/2606.23653/) (2026)
- [Open-KNEAD: Knowledge-grounded Nutrition Estimation via Agentic Decomposition](https://mlsanity.com/paper/2607.12911/) (2026)
- [Geometry-Enhanced Portion Estimation for Multimodal LLMs](https://mlsanity.com/paper/2607.16514/) (2026)

## Abstract (arXiv)

Image-based methods to analyze food images have alleviated the user burden and biases associated with traditional methods. However, accurate portion estimation remains a major challenge due to the loss of 3D information in the 2D representation of foods captured by smartphone cameras or wearable devices. In this paper, we propose a new framework to estimate both food volume and energy from 2D images by leveraging the power of 3D food models and physical reference in the eating scene. Our method estimates the pose of the camera and the food object in the input image and recreates the eating occasion by rendering an image of a 3D model of the food with the estimated poses. We also introduce a new dataset, SimpleFood45, which contains 2D images of 45 food items and associated annotations including food volume, weight, and energy. Our method achieves an average error of 31.10 kCal (17.67%) on this dataset, outperforming existing portion estimation methods. The dataset can be accessed at: https://lorenz.ecn.purdue.edu/~gvinod/simplefood45/ and the code can be accessed at: https://gitlab.com/viper-purdue/monocular-food-volume-3d