mlsanityData to 7 Oct 2026

DietDelta: A Vision-Language Approach for Dietary Assessment via Before-and-After Images

arXiv 2604.06352 ↗First posted 7 Apr 2026cs.CV cs.AI cs.MMcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate dietary assessment is critical for precision nutrition, yet most image-based methods rely on a single pre-consumption image and provide only coarse, meal-level estimates. These approaches cannot determine what was actually consumed and often require restrictive inputs such as depth sensing, multi-view imagery, or explicit segmentation. In this paper, we propose a simple vision-language framework for food-item-level nutritional analysis using paired before-and-after eating images. Instead of relying on rigid segmentation masks, our method leverages natural language prompts to localize specific food items and estimate their weight directly from a single RGB image. We further estimate food consumption by predicting weight differences between paired images using a two-stage training strategy. We evaluate our method on three publicly available datasets and demonstrate consistent improvements over existing approaches, establishing a strong baseline for before-and-after dietary image analysis.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 41 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

15 of 41 references.

References · 15

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  5. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  6. 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 389
  7. 2021Nutrition5k: Towards Automatic Nutritional Understanding of Generic Foodcited by 58
  8. 2017Two-view 3D Reconstruction for Food Volume Estimationcited by 18
  9. 2023An End-to-end Food Portion Estimation Framework Based on Shape Reconstruction from Monocular Imagecited by 15
  10. 2025DeepCode: Open Agentic Codingcited by 14
  11. 2024Food Portion Estimation via 3D Object Scalingcited by 12
  12. 2024MFP3D: Monocular Food Portion Estimation Leveraging 3D Point Cloudscited by 9
3 more
  1. 2021A Comprehensive Survey of Image-Based Food Recognition and Volume Estimation Methods for Dietary Assessmentcited by 7
  2. 2025Comprehensive Evaluation of Large Multimodal Models for Nutrition Analysis: A New Benchmark Enriched with Contextual Metadatacited by 3
  3. 2026Food Portion Estimation: From Pixels to Caloriescited by 3

Built on by

Not cited yet.