# Learning to Ground Objects for Robot Task and Motion Planning

arXiv: https://arxiv.org/abs/2202.06674 · HTML: https://mlsanity.com/paper/2202.06674/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2022.

## Built on by (most cited first, 2 of 2)

- [Grounding Classical Task Planners via Vision-Language Models](https://mlsanity.com/paper/2304.08587/) (2023)
- [DKPROMPT: Domain Knowledge Prompting Vision-Language Models for Open-World Planning](https://mlsanity.com/paper/2406.17659/) (2024)

## Abstract (arXiv)

Task and motion planning (TAMP) algorithms have been developed to help robots plan behaviors in discrete and continuous spaces. Robots face complex real-world scenarios, where it is hardly possible to model all objects or their physical properties for robot planning (e.g., in kitchens or shopping centers). In this paper, we define a new object-centric TAMP problem, where the TAMP robot does not know object properties (e.g., size and weight of blocks). We then introduce Task-Motion Object-Centric planning (bf TMOC), a grounded TAMP algorithm that learns to ground objects and their physical properties with a physics engine. TMOC is particularly useful for those tasks that involve dynamic complex robot-multi-object interactions that can hardly be modeled beforehand. We have demonstrated and evaluated TMOC in simulation and using a real robot. Results show that TMOC outperforms competitive baselines from the literature in cumulative utility.