# SHOP-VRB: A Visual Reasoning Benchmark for Object Perception

arXiv: https://arxiv.org/abs/2004.02673 · HTML: https://mlsanity.com/paper/2004.02673/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2020.

## Built on by (most cited first, 13 of 13)

- [StructFormer: Learning Spatial Structure for Language-Guided Semantic Rearrangement of Novel Objects](https://mlsanity.com/paper/2110.10189/) (2021)
- [Compositional Scene Representation Learning via Reconstruction: A Survey](https://mlsanity.com/paper/2202.07135/) (2022)
- [Interactive Learning of Physical Object Properties Through Robot Manipulation and Database of Object Measurements](https://mlsanity.com/paper/2404.07344/) (2024)
- [Unsupervised Object-Centric Learning from Multiple Unspecified Viewpoints](https://mlsanity.com/paper/2401.01922/) (2024)
- [Closed Loop Interactive Embodied Reasoning for Robot Manipulation](https://mlsanity.com/paper/2404.15194/) (2024)
- [Time-Conditioned Generative Modeling of Object-Centric Representations for Video Decomposition and Prediction](https://mlsanity.com/paper/2301.08951/) (2023)
- [Learning Global Object-Centric Representations via Disentangled Slot Attention](https://mlsanity.com/paper/2410.18809/) (2024)
- [Sim-To-Real Transfer of Visual Grounding for Human-Aided Ambiguity Resolution](https://mlsanity.com/paper/2205.12089/) (2022)
- [OCTScenes: A Versatile Real-World Dataset of Tabletop Scenes for Object-Centric Learning](https://mlsanity.com/paper/2306.09682/) (2023)
- [DetailCLIP: Injecting Image Details into CLIP's Feature Space](https://mlsanity.com/paper/2208.14649/) (2022)
- [Enhancing Interpretability and Interactivity in Robot Manipulation: A Neurosymbolic Approach](https://mlsanity.com/paper/2210.00858/) (2022)
- [Towards Unsupervised Visual Reasoning: Do Off-The-Shelf Features Know How to Reason?](https://mlsanity.com/paper/2212.10292/) (2022)
- [SAMPLE-HD: Simultaneous Action and Motion Planning Learning Environment](https://mlsanity.com/paper/2206.10312/) (2022)

## Abstract (arXiv)

In this paper we present an approach and a benchmark for visual reasoning in robotics applications, in particular small object grasping and manipulation. The approach and benchmark are focused on inferring object properties from visual and text data. It concerns small household objects with their properties, functionality, natural language descriptions as well as question-answer pairs for visual reasoning queries along with their corresponding scene semantic representations. We also present a method for generating synthetic data which allows to extend the benchmark to other objects or scenes and propose an evaluation protocol that is more challenging than in the existing datasets. We propose a reasoning system based on symbolic program execution. A disentangled representation of the visual and textual inputs is obtained and used to execute symbolic programs that represent a 'reasoning process' of the algorithm. We perform a set of experiments on the proposed benchmark and compare to results for the state of the art methods. These results expose the shortcomings of the existing benchmarks that may lead to misleading conclusions on the actual performance of the visual reasoning systems.