# Food Image Classification and Segmentation with Attention-based Multiple Instance Learning

arXiv: https://arxiv.org/abs/2308.11452 · HTML: https://mlsanity.com/paper/2308.11452/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 22 Aug 2023.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformers](https://mlsanity.com/paper/2012.15840/) (2020)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [CCNet: Criss-Cross Attention for Semantic Segmentation](https://mlsanity.com/paper/1811.11721/) (2018)
- [Panoptic Feature Pyramid Networks](https://mlsanity.com/paper/1901.02446/) (2019)
- [Large Scale Visual Food Recognition](https://mlsanity.com/paper/2103.16107/) (2021)
- [CAMEL: A Weakly Supervised Learning Framework for Histopathology Image Segmentation](https://mlsanity.com/paper/1908.10555/) (2019)
- [A Large-Scale Benchmark for Food Image Segmentation](https://mlsanity.com/paper/2105.05409/) (2021)
- [A Comprehensive Survey of Image-Based Food Recognition and Volume Estimation Methods for Dietary Assessment](https://mlsanity.com/paper/2106.11776/) (2021)
- [MyFood: A Food Segmentation and Classification System to Aid Nutritional Monitoring](https://mlsanity.com/paper/2012.03087/) (2020)
- [An Interpretable Multiple-Instance Approach for the Detection of referable Diabetic Retinopathy from Fundus Images](https://mlsanity.com/paper/2103.01702/) (2021)

## Built on by (most cited first, 1 of 1)

- [Weakly Supervised Food Image Segmentation using Vision Transformers and Segment Anything Model](https://mlsanity.com/paper/2509.19028/) (2025)

## Abstract (arXiv)

The demand for accurate food quantification has increased in the recent years, driven by the needs of applications in dietary monitoring. At the same time, computer vision approaches have exhibited great potential in automating tasks within the food domain. Traditionally, the development of machine learning models for these problems relies on training data sets with pixel-level class annotations. However, this approach introduces challenges arising from data collection and ground truth generation that quickly become costly and error-prone since they must be performed in multiple settings and for thousands of classes. To overcome these challenges, the paper presents a weakly supervised methodology for training food image classification and semantic segmentation models without relying on pixel-level annotations. The proposed methodology is based on a multiple instance learning approach in combination with an attention-based mechanism. At test time, the models are used for classification and, concurrently, the attention mechanism generates semantic heat maps which are used for food class segmentation. In the paper, we conduct experiments on two meta-classes within the FoodSeg103 data set to verify the feasibility of the proposed approach and we explore the functioning properties of the attention mechanism.