Can Large Language Models Reason about the Region Connection Calculus?
On the map: LLM reasoning → Spatial and embodied reasoning
Abstract (arXiv)
Qualitative Spatial Reasoning is a well explored area of Knowledge Representation and Reasoning and has multiple applications ranging from Geographical Information Systems to Robotics and Computer Vision. Recently, many claims have been made for the reasoning capabilities of Large Language Models (LLMs). Here, we investigate the extent to which a set of representative LLMs can perform classical qualitative spatial reasoning tasks on the mereotopological Region Connection Calculus, RCC-8. We conduct three pairs of experiments (reconstruction of composition tables, alignment to human composition preferences, conceptual neighbourhood reconstruction) using state-of-the-art LLMs; in each pair one experiment uses eponymous relations and one, anonymous relations (to test the extent to which the LLM relies on knowledge about the relation names obtained during training). All instances are repeated 30 times to measure the stochasticity of the LLMs.
Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 11 of its 42 references that are in the map. First posted on arXiv 29 Nov 2024.
Builds on
11 of 42 references.
Closest ancestors · 2
Same direction, newest first
- 2023An Evaluation of ChatGPT-4's Qualitative Spatial Reasoning Capabilities in RCC-827 Sep
- 2023Dialectical language model evaluation: An initial appraisal of the commonsense spatial reasoning abilities of LLMs22 Apr
Other references · 9
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2024The Prompt Report: A Systematic Survey of Prompt Engineering Techniquescited by 121
- 2023Evaluating Spatial Understanding of Large Language Modelscited by 66
- 2022How to Prompt? Opportunities and Challenges of Zero- and Few-Shot Learning for Human-AI Interaction in Creative Applications of Generative Modelscited by 44
- 2023Human-in-the-Loop through Chain-of-Thoughtcited by 20
- 2024Evaluating the Ability of Large Language Models to Reason about Cardinal Directionscited by 10
Built on by
Citing papers per year: 2025: 3; 2026: 2.
5 papers.
- 2025Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relationscited by 23
- 2025Large Language and Reasoning Models are Shallow Disjunctive Reasonerscited by 3
- 2025Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisitedcited by 1
- 2026QSTRBench: a New Benchmark to Evaluate the Ability of Language Models to Reason with Qualitative Spatial and Temporal Calculicited by 1
- 2026LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agentscited by 0
- 2026LMM Modality Transfer: A Pre-requisite for Autonomous GIS Agents7 Aug 2026
- 2026QSTRBench: a New Benchmark to Evaluate the Ability of Language Models to Reason with Qualitative Spatial and Temporal Calculi18 May 2026
- 2025Evaluating the Ability of Large Language Models to Reason about Cardinal Directions, Revisited16 Jul 2025
- 2025Foundation Models for Geospatial Reasoning: Assessing Capabilities of Large Language Models in Understanding Geometries and Topological Spatial Relations22 May 2025
- 2025Large Language and Reasoning Models are Shallow Disjunctive Reasoners30 Mar 2025
Similar, not cited
Close in wording, no citation either way
- 2024Can Large Language Models Create New Knowledge for Spatial Reasoning Tasks?
- 2026From Symbolic to Geometric: Enabling Spatial Reasoning in Large Language Models
- 2024Reframing Spatial Reasoning Evaluation in Language Models: A Real-World Simulation Benchmark for Qualitative Reasoning
- 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
- 2024SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models
- 2025LTD-Bench: Evaluating Large Language Models by Letting Them Draw
- 2026From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
- 2024Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models