# Exploiting Embodied Simulation to Detect Novel Object Classes Through Interaction

arXiv: https://arxiv.org/abs/2204.08107 · HTML: https://mlsanity.com/paper/2204.08107/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 49 references that are in the map.
- First posted on arXiv 17 Apr 2022.

## Builds on (most cited first, 10 of 10)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [It's Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners](https://mlsanity.com/paper/2009.07118/) (2020)
- [Unity: A General Platform for Intelligent Agents](https://mlsanity.com/paper/1809.02627/) (2018)
- [Learning Physical Intuition of Block Towers by Example](https://mlsanity.com/paper/1603.01312/) (2016)
- [Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning](https://mlsanity.com/paper/1912.11032/) (2019)
- [VoxML: A Visualization Modeling Language](https://mlsanity.com/paper/1610.01508/) (2016)
- [Acquiring Target Stacking Skills by Goal-Parameterized Deep Reinforcement Learning](https://mlsanity.com/paper/1711.00267/) (2017)

## Built on by (most cited first, 2 of 2)

- [Automatically Discovering Novel Visual Categories with Self-supervised Prototype Learning](https://mlsanity.com/paper/2208.00979/) (2022)
- [Detecting and Accommodating Novel Types and Concepts in an Embodied Simulation Environment](https://mlsanity.com/paper/2211.04555/) (2022)

## Abstract (arXiv)

In this paper we present a novel method for a naive agent to detect novel objects it encounters in an interaction. We train a reinforcement learning policy on a stacking task given a known object type, and then observe the results of the agent attempting to stack various other objects based on the same trained policy. By extracting embedding vectors from a convolutional neural net trained over the results of the aforementioned stacking play, we can determine the similarity of a given object to known object types, and determine if the given object is likely dissimilar enough to the known types to be considered a novel class of object. We present the results of this method on two datasets gathered using two different policies and demonstrate what information the agent needs to extract from its environment to make these novelty judgments.