# Functional Object-Oriented Network for Manipulation Learning

arXiv: https://arxiv.org/abs/1902.01537 · HTML: https://mlsanity.com/paper/1902.01537/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2019.

## Built on by (most cited first, 27 of 27)

- [EgoTaskQA: Understanding Human Tasks in Egocentric Videos](https://mlsanity.com/paper/2210.03929/) (2022)
- [GenDP: 3D Semantic Fields for Category-Level Generalizable Diffusion Policy](https://mlsanity.com/paper/2410.17488/) (2024)
- [ManipDreamer: Boosting Robotic Manipulation World Model with Action Tree and Visual Guidance](https://mlsanity.com/paper/2504.16464/) (2025)
- [From Cooking Recipes to Robot Task Trees -- Improving Planning Correctness and Task Efficiency by Leveraging LLMs with a Knowledge Network](https://mlsanity.com/paper/2309.09181/) (2023)
- [Evaluating Recipes Generated from Functional Object-Oriented Network](https://mlsanity.com/paper/2106.00728/) (2021)
- [Generalizable task representation learning from human demonstration videos: a geometric approach](https://mlsanity.com/paper/2202.13604/) (2022)
- [Long-Horizon Planning and Execution with Functional Object-Oriented Networks](https://mlsanity.com/paper/2207.05800/) (2022)
- [Learning Part-Aware Dense 3D Feature Field for Generalizable Articulated Object Manipulation](https://mlsanity.com/paper/2602.14193/) (2026)
- [AdaptBot: Combining LLM with Knowledge Graphs and Human Input for Generic-to-Specific Task Decomposition and Knowledge Refinement](https://mlsanity.com/paper/2502.02067/) (2025)
- [STAR: A Foundation Model-driven Framework for Robust Task Planning and Failure Recovery in Robotic Systems](https://mlsanity.com/paper/2503.06060/) (2025)
- [Cooking Object's State Identification Without Using Pretrained Model](https://mlsanity.com/paper/2103.02305/) (2021)
- [Pouring Dynamics Estimation Using Gated Recurrent Units](https://mlsanity.com/paper/2105.12828/) (2021)
- [A Road-map to Robot Task Execution with the Functional Object-Oriented Network](https://mlsanity.com/paper/2106.00158/) (2021)
- [Functional Task Tree Generation from a Knowledge Graph to Solve Unseen Problems](https://mlsanity.com/paper/2112.02433/) (2021)
- [Classifying States of Cooking Objects Using Convolutional Neural Network](https://mlsanity.com/paper/2105.14196/) (2021)
- [Calorie Aware Automatic Meal Kit Generation from an Image](https://mlsanity.com/paper/2112.09839/) (2021)
- [Grounding of the Functional Object-Oriented Network in Industrial Tasks](https://mlsanity.com/paper/2204.02274/) (2022)
- [FOON Creation and Traversal for Recipe Generation](https://mlsanity.com/paper/2210.07335/) (2022)
- [Task Tree Retrieval for Robotic Cooking](https://mlsanity.com/paper/2211.01745/) (2022)
- [Knowledge Retrieval using Functional Object-Oriented Network](https://mlsanity.com/paper/2211.03037/) (2022)
- [Knowledge Retrieval using Foon](https://mlsanity.com/paper/2211.03790/) (2022)
- [Knowledge Retrieval for Robotic Cooking](https://mlsanity.com/paper/2211.04524/) (2022)
- [Task Tree Retrieval Algorithms for Robotic Cooking Using The Functional Object-Oriented Network](https://mlsanity.com/paper/2211.06743/) (2022)
- [Knowledge Retrieval Using Functional Object-Oriented Networks](https://mlsanity.com/paper/2211.14896/) (2022)
- [Task tree retrieval from FOON using search algorithms](https://mlsanity.com/paper/2401.05346/) (2023)
- [Prompting Task Trees using Gemini: Methodologies and Insights](https://mlsanity.com/paper/2405.03671/) (2024)
- [Knowledge Graph Completion for Action Prediction on Situational Graphs -- A Case Study on Household Tasks](https://mlsanity.com/paper/2508.13675/) (2025)

## Abstract (arXiv)

This paper presents a novel structured knowledge representation called the functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects. Using a well-trained FOON, robots can decipher a task goal, seek the correct objects at the desired states on which to operate, and generate a sequence of proper manipulation motions. The paper describes FOON's structure and an approach to form a universal FOON with extracted knowledge from online instructional videos. A graph retrieval approach is presented to generate manipulation motion sequences from the FOON to achieve a desired goal, demonstrating the flexibility of FOON in creating a novel and adaptive means of solving a problem using knowledge gathered from multiple sources. The results are demonstrated in a simulated environment to illustrate the motion sequences generated from the FOON to carry out the desired tasks.