# GRASP: A Grid-Based Benchmark for Evaluating Commonsense Spatial Reasoning

arXiv: https://arxiv.org/abs/2407.01892 · HTML: https://mlsanity.com/paper/2407.01892/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Minigrid & Miniworld: Modular & Customizable Reinforcement Learning Environments for Goal-Oriented Tasks](https://mlsanity.com/paper/2306.13831/) (2023)
- [BabyAI: A Platform to Study the Sample Efficiency of Grounded Language Learning](https://mlsanity.com/paper/1810.08272/) (2018)
- [Large Language Models as Commonsense Knowledge for Large-Scale Task Planning](https://mlsanity.com/paper/2305.14078/) (2023)
- [Large Language Models and Games: A Survey and Roadmap](https://mlsanity.com/paper/2402.18659/) (2024)
- [I-Design: Personalized LLM Interior Designer](https://mlsanity.com/paper/2404.02838/) (2024)
- [SpartQA: : A Textual Question Answering Benchmark for Spatial Reasoning](https://mlsanity.com/paper/2104.05832/) (2021)
- [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://mlsanity.com/paper/2204.08292/) (2022)
- [Evaluating Spatial Understanding of Large Language Models](https://mlsanity.com/paper/2310.14540/) (2023)
- [Do Language Embeddings Capture Scales?](https://mlsanity.com/paper/2010.05345/) (2020)
- [A Systematic Investigation of Commonsense Knowledge in Large Language Models](https://mlsanity.com/paper/2111.00607/) (2021)
- [Can Large Language Models be Good Path Planners? A Benchmark and Investigation on Spatial-temporal Reasoning](https://mlsanity.com/paper/2310.03249/) (2023)
- [Things not Written in Text: Exploring Spatial Commonsense from Visual Signals](https://mlsanity.com/paper/2203.08075/) (2022)
- [Exploring and Improving the Spatial Reasoning Abilities of Large Language Models](https://mlsanity.com/paper/2312.01054/) (2023)
- [Commonsense Reasoning for Conversational AI: A Survey of the State of the Art](https://mlsanity.com/paper/2302.07926/) (2023)
- [A Survey of Generative AI for Intelligent Transportation Systems: Road Transportation Perspective](https://mlsanity.com/paper/2312.08248/) (2023)
- [Beyond Language: Learning Commonsense from Images for Reasoning](https://mlsanity.com/paper/2010.05001/) (2020)

## Built on by (most cited first, 7 of 7)

- [VideoCogQA: A Controllable Benchmark for Evaluating Cognitive Abilities in Video-Language Models](https://mlsanity.com/paper/2411.09105/) (2024)
- [LRR-Bench: Left, Right or Rotate? Vision-Language models Still Struggle With Spatial Understanding Tasks](https://mlsanity.com/paper/2507.20174/) (2025)
- [Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience](https://mlsanity.com/paper/2505.05515/) (2025)
- [seqBench: A Tunable Benchmark to Quantify Sequential Reasoning Limits of LLMs](https://mlsanity.com/paper/2509.16866/) (2025)
- [CartoMapQA: A Fundamental Benchmark Dataset Evaluating Vision-Language Models on Cartographic Map Understanding](https://mlsanity.com/paper/2512.03558/) (2025)
- [Large Language Models and their Awareness of Mechanics and Spatial Geometry](https://mlsanity.com/paper/2608.14615/) (2026)
- [The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space](https://mlsanity.com/paper/2605.09883/) (2026)

## Abstract (arXiv)

Spatial reasoning, an important faculty of human cognition with many practical applications, is one of the core commonsense skills that is not purely language-based and, for satisfying (as opposed to optimal) solutions, requires some minimum degree of planning. Existing benchmarks of Commonsense Spatial Reasoning (CSR) tend to evaluate how Large Language Models (LLMs) interpret text-based spatial textitdescriptions rather than directly evaluate a plan produced by the LLM in response to a textitspecific spatial reasoning problem. In this paper, we construct a large-scale benchmark called GRASP, which consists of 16,000 grid-based environments where the agent is tasked with an energy collection problem. These environments include 100 grid instances instantiated using each of the 160 different grid settings, involving five different energy distributions, two modes of agent starting position, and two distinct obstacle configurations, as well as three kinds of agent constraints. Using GRASP, we compare classic baseline approaches, such as random walk and greedy search methods, with advanced LLMs like GPT-3.5-Turbo, GPT-4o, and GPT-o1-mini. The experimental results indicate that even these advanced LLMs struggle to consistently achieve satisfactory solutions.