# Deep Learning based Food Instance Segmentation using Synthetic Data

arXiv: https://arxiv.org/abs/2107.07191 · HTML: https://mlsanity.com/paper/2107.07191/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 38 references that are in the map.
- First posted on arXiv 15 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [FoodX-251: A Dataset for Fine-grained Food Classification](https://mlsanity.com/paper/1907.06167/) (2019)
- [ChineseFoodNet: A large-scale Image Dataset for Chinese Food Recognition](https://mlsanity.com/paper/1705.02743/) (2017)
- [Simultaneous Food Localization and Recognition](https://mlsanity.com/paper/1604.07953/) (2016)
- [MUSEFood: Multi-sensor-based Food Volume Estimation on Smartphones](https://mlsanity.com/paper/1903.07437/) (2019)
- [Segmenting Unseen Industrial Components in a Heavy Clutter Using RGB-D Fusion and Synthetic Data](https://mlsanity.com/paper/2002.03501/) (2020)

## Built on by (most cited first, 3 of 3)

- [Cluttered Food Grasping with Adaptive Fingers and Synthetic-Data Trained Object Detection](https://mlsanity.com/paper/2203.05187/) (2022)
- [A Dual-Cycled Cross-View Transformer Network for Unified Road Layout Estimation and 3D Object Detection in the Bird's-Eye-View](https://mlsanity.com/paper/2209.08844/) (2022)
- [Artificial Intelligence in the Food Industry: Food Waste Estimation based on Computer Vision, a Brief Case Study in a University Dining Hall](https://mlsanity.com/paper/2507.14662/) (2025)

## Abstract (arXiv)

In the process of intelligently segmenting foods in images using deep neural networks for diet management, data collection and labeling for network training are very important but labor-intensive tasks. In order to solve the difficulties of data collection and annotations, this paper proposes a food segmentation method applicable to real-world through synthetic data. To perform food segmentation on healthcare robot systems, such as meal assistance robot arm, we generate synthetic data using the open-source 3D graphics software Blender placing multiple objects on meal plate and train Mask R-CNN for instance segmentation. Also, we build a data collection system and verify our segmentation model on real-world food data. As a result, on our real-world dataset, the model trained only synthetic data is available to segment food instances that are not trained with 52.2% mask AP@all, and improve performance by +6.4%p after fine-tuning comparing to the model trained from scratch. In addition, we also confirm the possibility and performance improvement on the public dataset for fair analysis. Our code and pre-trained weights are avaliable online at: https://github.com/gist-ailab/Food-Instance-Segmentation