mlsanityData to 7 Oct 2026

An Evaluation of ChatGPT-4's Qualitative Spatial Reasoning Capabilities in RCC-8

arXiv 2309.15577 ↗First posted 27 Sep 2023cs.AIcited by 9

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Qualitative Spatial Reasoning (QSR) is well explored area of Commonsense Reasoning and has multiple applications ranging from Geographical Information Systems to Robotics and Computer Vision. Recently many claims have been made for the capabilities of Large Language Models (LLMs). In this paper we investigate the extent to which one particular LLM can perform classical qualitative spatial reasoning tasks on the mereotopological calculus, RCC-8.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 7 of its 21 references that are in the map. First posted on arXiv 27 Sep 2023.

Builds on

7 of 21 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs22 Apr

Other references · 6

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  4. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  5. 2022How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Modelscited by 43
  6. 2023Human-in-the-Loop through Chain-of-Thoughtcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 2; 2026: 2.

9 papers.

  1. 2025Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relationscited by 20
  2. 2024Evaluating the Ability of Large Language Models to Reason about Cardinal Directionscited by 10
  3. 2024Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMscited by 6
  4. 2024Distortions in Judged Spatial Relations in Large Language Modelscited by 5
  5. 2024Can Large Language Models Reason about the Region Connection Calculus?cited by 5
  6. 2024PLUGH: A Benchmark for Spatial Understanding and Reasoning in Large Language Modelscited by 2
  7. 2025Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisitedcited by 1
  8. 2026QSTRBench: a New Benchmark to Evaluate the Ability of Language Models to Reason with Qualitative Spatial and Temporal Calculicited by 0
  9. 2026Spatial Reasoning via Modality Switching Between Language and Symbolic Representationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning
  2. 2024Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark
  3. 2026From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models
  4. 2024SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models
  5. 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
  6. 2024Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?
  7. 2025Scaling and Beyond: Advancing Spatial Reasoning in MLLMs Requires New Recipes
  8. 2025Open3D-VQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space