# Functional Task Tree Generation from a Knowledge Graph to Solve Unseen Problems

arXiv: https://arxiv.org/abs/2112.02433 · HTML: https://mlsanity.com/paper/2112.02433/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 25 references that are in the map.
- First posted on arXiv 4 Dec 2021.

## Builds on (most cited first, 9 of 9)

- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Recipe1M+: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images](https://mlsanity.com/paper/1810.06553/) (2018)
- [Functional Object-Oriented Network for Manipulation Learning](https://mlsanity.com/paper/1902.01537/) (2019)
- [A Survey of Knowledge Representation in Service Robotics](https://mlsanity.com/paper/1807.02192/) (2018)
- [Functional Object-Oriented Network: Construction & Expansion](https://mlsanity.com/paper/1807.02189/) (2018)
- [Joint Object and State Recognition using Language Knowledge](https://mlsanity.com/paper/1905.08843/) (2019)
- [Long Activity Video Understanding using Functional Object-Oriented Network](https://mlsanity.com/paper/1807.00983/) (2018)
- [Identifying Object States in Cooking-Related Images](https://mlsanity.com/paper/1805.06956/) (2018)
- [Evaluating Recipes Generated from Functional Object-Oriented Network](https://mlsanity.com/paper/2106.00728/) (2021)

## Built on by (most cited first, 1 of 1)

- [Calorie Aware Automatic Meal Kit Generation from an Image](https://mlsanity.com/paper/2112.09839/) (2021)

## Abstract (arXiv)

A major component for developing intelligent and autonomous robots is a suitable knowledge representation, from which a robot can acquire knowledge about its actions or world. However, unlike humans, robots cannot creatively adapt to novel scenarios, as their knowledge and environment are rigidly defined. To address the problem of producing novel and flexible task plans called task trees, we explore how we can derive plans with concepts not originally in the robot's knowledge base. Existing knowledge in the form of a knowledge graph is used as a base of reference to create task trees that are modified with new object or state combinations. To demonstrate the flexibility of our method, we randomly selected recipes from the Recipe1M+ dataset and generated their task trees. The task trees were then thoroughly checked with a visualization tool that portrays how each ingredient changes with each action to produce the desired meal. Our results indicate that the proposed method can produce task plans with high accuracy even for never-before-seen ingredient combinations.