# Identifying Object States in Cooking-Related Images

arXiv: https://arxiv.org/abs/1805.06956 · HTML: https://mlsanity.com/paper/1805.06956/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 May 2018.

## Built on by (most cited first, 7 of 7)

- [Rethinking Cooking State Recognition with Vision Transformers](https://mlsanity.com/paper/2212.08586/) (2022)
- [Details Make a Difference: Object State-Sensitive Neurorobotic Task Planning](https://mlsanity.com/paper/2406.09988/) (2024)
- [Continuous Object State Recognition for Cooking Robots Using Pre-Trained Vision-Language Models and Black-box Optimization](https://mlsanity.com/paper/2403.08239/) (2024)
- [Cooking Object's State Identification Without Using Pretrained Model](https://mlsanity.com/paper/2103.02305/) (2021)
- [Functional Task Tree Generation from a Knowledge Graph to Solve Unseen Problems](https://mlsanity.com/paper/2112.02433/) (2021)
- [Classifying States of Cooking Objects Using Convolutional Neural Network](https://mlsanity.com/paper/2105.14196/) (2021)
- [Calorie Aware Automatic Meal Kit Generation from an Image](https://mlsanity.com/paper/2112.09839/) (2021)

## Abstract (arXiv)

Understanding object states is as important as object recognition for robotic task planning and manipulation. To our knowledge, this paper explicitly introduces and addresses the state identification problem in cooking related images for the first time. In this paper, objects and ingredients in cooking videos are explored and the most frequent objects are analyzed. Eleven states from the most frequent cooking objects are examined and a dataset of images containing those objects and their states is created. As a solution to the state identification problem, a Resnet based deep model is proposed. The model is initialized with Imagenet weights and trained on the dataset of eleven classes. The trained state identification model is evaluated on a subset of the Imagenet dataset and state labels are provided using a combination of the model with manual checking. Moreover, an individual model is fine-tuned for each object in the dataset using the weights from the initially trained model and object-specific images, where significant improvement is demonstrated.