# Commonsense Spatial Reasoning for Visually Intelligent Agents

arXiv: https://arxiv.org/abs/2104.00387 · HTML: https://mlsanity.com/paper/2104.00387/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 32 references that are in the map.
- First posted on arXiv 1 Apr 2021.

## Builds on (most cited first, 7 of 7)

- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Robotic Pick-and-Place of Novel Objects in Clutter with Multi-Affordance Grasping and Cross-Domain Image Matching](https://mlsanity.com/paper/1710.01330/) (2017)
- [Recent Advances in Natural Language Inference: A Survey of Benchmarks, Resources, and Approaches](https://mlsanity.com/paper/1904.01172/) (2019)
- [SpatialSense: An Adversarially Crowdsourced Benchmark for Spatial Relation Recognition](https://mlsanity.com/paper/1908.02660/) (2019)
- [Towards a Framework for Visual Intelligence in Service Robotics: Epistemic Requirements and Gap Analysis](https://mlsanity.com/paper/2003.06171/) (2020)
- [Fit to Measure: Reasoning about Sizes for Robust Object Recognition](https://mlsanity.com/paper/2010.14296/) (2020)
- [Where is My Stuff? An Interactive System for Spatial Relations](https://mlsanity.com/paper/1909.06331/) (2019)

## Abstract (arXiv)

Service robots are expected to reliably make sense of complex, fast-changing environments. From a cognitive standpoint, they need the appropriate reasoning capabilities and background knowledge required to exhibit human-like Visual Intelligence. In particular, our prior work has shown that the ability to reason about spatial relations between objects in the world is a key requirement for the development of Visually Intelligent Agents. In this paper, we present a framework for commonsense spatial reasoning which is tailored to real-world robotic applications. Differently from prior approaches to qualitative spatial reasoning, the proposed framework is robust to variations in the robot's viewpoint and object orientation. The spatial relations in the proposed framework are also mapped to the types of commonsense predicates used to describe typical object configurations in English. In addition, we also show how this formally-defined framework can be implemented in a concrete spatial database.