# OmniFood8K: Single-Image Nutrition Estimation via Hierarchical Frequency-Aligned Fusion

arXiv: https://arxiv.org/abs/2604.12356 · HTML: https://mlsanity.com/paper/2604.12356/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 72 references that are in the map.
- First posted on arXiv 14 Apr 2026.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Towards Automatic Learning of Procedures from Web Instructional Videos](https://mlsanity.com/paper/1703.09788/) (2017)
- [CMX: Cross-Modal Fusion for RGB-X Semantic Segmentation with Transformers](https://mlsanity.com/paper/2203.04838/) (2022)
- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [A Survey on Food Computing](https://mlsanity.com/paper/1808.07202/) (2018)
- [Nutrition5k: Towards Automatic Nutritional Understanding of Generic Food](https://mlsanity.com/paper/2103.03375/) (2021)
- [ISIA Food-500: A Dataset for Large-Scale Food Recognition via Stacked Global-Local Attention Network](https://mlsanity.com/paper/2008.05655/) (2020)
- [A Large-Scale Benchmark for Food Image Segmentation](https://mlsanity.com/paper/2105.05409/) (2021)
- [TriTransNet: RGB-D Salient Object Detection with a Triplet Transformer Embedding Network](https://mlsanity.com/paper/2108.03990/) (2021)
- [Cross-modality Discrepant Interaction Network for RGB-D Salient Object Detection](https://mlsanity.com/paper/2108.01971/) (2021)
- [An End-to-end Food Portion Estimation Framework Based on Shape Reconstruction from Monocular Image](https://mlsanity.com/paper/2308.01810/) (2023)
- [RoDE: Linear Rectified Mixture of Diverse Experts for Food Large Multi-Modal Models](https://mlsanity.com/paper/2407.12730/) (2024)
- [Image Based Food Energy Estimation With Depth Domain Adaptation](https://mlsanity.com/paper/2208.12153/) (2022)
- [DPF-Nutrition: Food Nutrition Estimation via Depth Prediction and Fusion](https://mlsanity.com/paper/2310.11702/) (2023)
- [A review on vision-based analysis for automatic dietary assessment](https://mlsanity.com/paper/2108.02947/) (2021)
- [Navigating Weight Prediction with Diet Diary](https://mlsanity.com/paper/2408.05445/) (2024)
- [Are Vision-Language Models Ready for Dietary Assessment? Exploring the Next Frontier in AI-Powered Food Image Recognition](https://mlsanity.com/paper/2504.06925/) (2025)
- [Advancing Food Nutrition Estimation via Visual-Ingredient Feature Fusion](https://mlsanity.com/paper/2505.08747/) (2025)
- [Joint Geometric and Trajectory Consistency Learning for One-Step Real-World Super-Resolution](https://mlsanity.com/paper/2602.24240/) (2026)
- [FMiFood: Multi-modal Contrastive Learning for Food Image Classification](https://mlsanity.com/paper/2408.03922/) (2024)
- [Explainable Artificial Intelligence Techniques for Interpretation of Food Models: a Review](https://mlsanity.com/paper/2504.10527/) (2025)

## Built on by (most cited first, 2 of 2)

- [Open-KNEAD: Knowledge-grounded Nutrition Estimation via Agentic Decomposition](https://mlsanity.com/paper/2607.12911/) (2026)
- [NutriVision: Ingredient-Conditioned Fusion and Prediction for Single-Image Food Nutrition Estimation](https://mlsanity.com/paper/2609.33076/) (2026)

## Abstract (arXiv)

Accurate estimation of food nutrition plays a vital role in promoting healthy dietary habits and personalized diet management. Most existing food datasets primarily focus on Western cuisines and lack sufficient coverage of Chinese dishes, which restricts accurate nutritional estimation for Chinese meals. Moreover, many state-of-the-art nutrition prediction methods rely on depth sensors, restricting their applicability in daily scenarios. To address these limitations, we introduce OmniFood8K, a comprehensive multimodal dataset comprising 8,036 food samples, each with detailed nutritional annotations and multi-view images. In addition, to enhance models' capability in nutritional prediction, we construct NutritionSynth-115K, a large-scale synthetic dataset that introduces compositional variations while preserving precise nutritional labels. Moreover, we propose an end-to-end framework for nutritional prediction from a single RGB image. First, we predict a depth map from a single RGB image and design the Scale-Shift Residual Adapter (SSRA) to refine it for global scale consistency and local structural preservation. Second, we propose the Frequency-Aligned Fusion Module (FAFM) to hierarchically align and fuse RGB and depth features in the frequency domain. Finally, we design a Mask-based Prediction Head (MPH) to emphasize key ingredient regions via dynamic channel selection for more accurate prediction. Extensive experiments on multiple datasets demonstrate the superiority of our method over existing approaches. Project homepage: https://yudongjian.github.io/OmniFood8K-food/