# An Evaluation of ChatGPT-4's Qualitative Spatial Reasoning Capabilities in RCC-8

arXiv: https://arxiv.org/abs/2309.15577 · HTML: https://mlsanity.com/paper/2309.15577/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 27 Sep 2023.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 7 of 7)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Models](https://mlsanity.com/paper/2209.01390/) (2022)
- [Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs](https://mlsanity.com/paper/2304.11164/) (2023)
- [Human-in-the-Loop through Chain-of-Thought](https://mlsanity.com/paper/2306.07932/) (2023)

## Built on by (most cited first, 9 of 9)

- [Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations](https://mlsanity.com/paper/2505.17136/) (2025)
- [Evaluating the Ability of Large Language Models to Reason about Cardinal Directions](https://mlsanity.com/paper/2406.16528/) (2024)
- [Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs](https://mlsanity.com/paper/2411.18564/) (2024)
- [Distortions in Judged Spatial Relations in Large Language Models](https://mlsanity.com/paper/2401.04218/) (2024)
- [Can Large Language Models Reason about the Region Connection Calculus?](https://mlsanity.com/paper/2411.19589/) (2024)
- [PLUGH: A Benchmark for Spatial Understanding and Reasoning in Large Language Models](https://mlsanity.com/paper/2408.04648/) (2024)
- [Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited](https://mlsanity.com/paper/2507.12059/) (2025)
- [QSTRBench: a New Benchmark to Evaluate the Ability of Language Models to Reason with Qualitative Spatial and Temporal Calculi](https://mlsanity.com/paper/2605.18380/) (2026)
- [Spatial Reasoning via Modality Switching Between Language and Symbolic Representations](https://mlsanity.com/paper/2606.31285/) (2026)

## Abstract (arXiv)

Qualitative Spatial Reasoning (QSR) is well explored area of Commonsense Reasoning and has multiple applications ranging from Geographical Information Systems to Robotics and Computer Vision. Recently many claims have been made for the capabilities of Large Language Models (LLMs). In this paper we investigate the extent to which one particular LLM can perform classical qualitative spatial reasoning tasks on the mereotopological calculus, RCC-8.