# Put the Bear on the Chair! Intelligent Robot Interaction with Previously Unseen Chairs via Robot Imagination

arXiv: https://arxiv.org/abs/2108.05539 · HTML: https://mlsanity.com/paper/2108.05539/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 43 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 5 of 5)

- [AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection](https://mlsanity.com/paper/1709.07326/) (2017)
- [Detecting Human-Object Interactions via Functional Generalization](https://mlsanity.com/paper/1904.03181/) (2019)
- [Is That a Chair? Imagining Affordances Using Simulations of an Articulated Human Body](https://mlsanity.com/paper/1909.07572/) (2019)
- [Can I Pour into It? Robot Imagining Open Containability Affordance of Previously Unseen Objects via Physical Simulations](https://mlsanity.com/paper/2008.02321/) (2020)
- [Can I lift it? Humanoid robot reasoning about the feasibility of lifting a heavy box with unknown physical properties](https://mlsanity.com/paper/2008.03801/) (2020)

## Abstract (arXiv)

In this paper, we study the problem of autonomously seating a teddy bear on a previously unseen chair. To achieve this goal, we present a novel method for robots to imagine the sitting pose of the bear by physically simulating a virtual humanoid agent sitting on the chair. We also develop a robotic system which leverages motion planning to plan SE(2) motions for a humanoid robot to walk to the chair and whole-body motions to put the bear on it. Furthermore, to cope with cases where the chair is not in an accessible pose for placing the bear, a human assistance module is introduced for a human to follow language instructions given by the robot to rotate the chair and help make the chair accessible. We implement our method with a robot arm and a humanoid robot. We calibrate the proposed system with 3 chairs and test on 12 previously unseen chairs in both accessible and inaccessible poses extensively. Results show that our method enables the robot to autonomously seat the teddy bear on the 12 previously unseen chairs with a very high success rate. The human assistance module is also shown to be very effective in changing the accessibility of the chair. Video demos and more details are available at https://chirikjianlab.github.io/putbearonchair/.