# Visual Prediction of Priors for Articulated Object Interaction

arXiv: https://arxiv.org/abs/2006.03979 · HTML: https://mlsanity.com/paper/2006.03979/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2020.

## Built on by (most cited first, 4 of 4)

- [PhyPlan: Compositional and Adaptive Physical Task Reasoning with Physics-Informed Skill Networks for Robot Manipulators](https://mlsanity.com/paper/2402.15767/) (2024)
- [Structured Latent Variable Models for Articulated Object Interaction](https://mlsanity.com/paper/2305.16567/) (2023)
- [PhyPlan: Generalizable and Rapid Physical Task Planning with Physics Informed Skill Networks for Robot Manipulators](https://mlsanity.com/paper/2406.00001/) (2024)
- [Semi-Supervised Neural Processes for Articulated Object Interactions](https://mlsanity.com/paper/2412.00145/) (2024)

## Abstract (arXiv)

Exploration in novel settings can be challenging without prior experience in similar domains. However, humans are able to build on prior experience quickly and efficiently. Children exhibit this behavior when playing with toys. For example, given a toy with a yellow and blue door, a child will explore with no clear objective, but once they have discovered how to open the yellow door, they will most likely be able to open the blue door much faster. Adults also exhibit this behavior when entering new spaces such as kitchens. We develop a method, Contextual Prior Prediction, which provides a means of transferring knowledge between interactions in similar domains through vision. We develop agents that exhibit exploratory behavior with increasing efficiency, by learning visual features that are shared across environments, and how they correlate to actions. Our problem is formulated as a Contextual Multi-Armed Bandit where the contexts are images, and the robot has access to a parameterized action space. Given a novel object, the objective is to maximize reward with few interactions. A domain which strongly exhibits correlations between visual features and motion is kinemetically constrained mechanisms. We evaluate our method on simulated prismatic and revolute joints.