# Acquiring Common Sense Spatial Knowledge through Implicit Spatial Templates

arXiv: https://arxiv.org/abs/1711.06821 · HTML: https://mlsanity.com/paper/1711.06821/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2017.

## Built on by (most cited first, 10 of 10)

- [Visual Spatial Reasoning](https://mlsanity.com/paper/2205.00363/) (2022)
- [Things not Written in Text: Exploring Spatial Commonsense from Visual Signals](https://mlsanity.com/paper/2203.08075/) (2022)
- [Not All Relations are Equal: Mining Informative Labels for Scene Graph Generation](https://mlsanity.com/paper/2111.13517/) (2021)
- [VCD: A Dataset for Visual Commonsense Discovery in Images](https://mlsanity.com/paper/2402.17213/) (2024)
- [MineAnyBuild: Benchmarking Spatial Planning for Open-world AI Agents](https://mlsanity.com/paper/2505.20148/) (2025)
- [Interpretation modeling: Social grounding of sentences by reasoning over their implicit moral judgments](https://mlsanity.com/paper/2312.03726/) (2023)
- [Inferring spatial relations from textual descriptions of images](https://mlsanity.com/paper/2102.00997/) (2021)
- [Visually Grounded Commonsense Knowledge Acquisition](https://mlsanity.com/paper/2211.12054/) (2022)
- [Grounding Spatial Relations in Text-Only Language Models](https://mlsanity.com/paper/2403.13666/) (2024)
- [GuessingGame: Measuring the Informativeness of Open-Ended Questions in Large Language Models](https://mlsanity.com/paper/2509.19593/) (2025)

## Abstract (arXiv)

Spatial understanding is a fundamental problem with wide-reaching real-world applications. The representation of spatial knowledge is often modeled with spatial templates, i.e., regions of acceptability of two objects under an explicit spatial relationship (e.g., "on", "below", etc.). In contrast with prior work that restricts spatial templates to explicit spatial prepositions (e.g., "glass on table"), here we extend this concept to implicit spatial language, i.e., those relationships (generally actions) for which the spatial arrangement of the objects is only implicitly implied (e.g., "man riding horse"). In contrast with explicit relationships, predicting spatial arrangements from implicit spatial language requires significant common sense spatial understanding. Here, we introduce the task of predicting spatial templates for two objects under a relationship, which can be seen as a spatial question-answering task with a (2D) continuous output ("where is the man w.r.t. a horse when the man is walking the horse?"). We present two simple neural-based models that leverage annotated images and structured text to learn this task. The good performance of these models reveals that spatial locations are to a large extent predictable from implicit spatial language. Crucially, the models attain similar performance in a challenging generalized setting, where the object-relation-object combinations (e.g.,"man walking dog") have never been seen before. Next, we go one step further by presenting the models with unseen objects (e.g., "dog"). In this scenario, we show that leveraging word embeddings enables the models to output accurate spatial predictions, proving that the models acquire solid common sense spatial knowledge allowing for such generalization.