# From Building Blocks to Planning: Multi-Step Spatial Reasoning in LLMs with Reinforcement Learning

arXiv: https://arxiv.org/abs/2512.24532 · HTML: https://mlsanity.com/paper/2512.24532/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning.
- Builds on 18 of its 29 references that are in the map.
- First posted on arXiv 31 Dec 2025.

## Placed in

- LLM reasoning / [Spatial and embodied reasoning](https://mlsanity.com/lr/spatial-and-embodied-reasoning/)

## Builds on (most cited first, 18 of 18)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Tree of Thoughts: Deliberate Problem Solving with Large Language Models](https://mlsanity.com/paper/2305.10601/) (2023)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [Large Language Models for Mathematical Reasoning: Progresses and Challenges](https://mlsanity.com/paper/2402.00157/) (2024)
- [Why Is Spatial Reasoning Hard for VLMs? An Attention Mechanism Perspective on Focus Areas](https://mlsanity.com/paper/2503.01773/) (2025)
- [Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Models](https://mlsanity.com/paper/2407.07035/) (2024)
- [Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2404.03622/) (2024)
- [A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges](https://mlsanity.com/paper/2501.02189/) (2025)
- [Grounding Multimodal Large Language Models in Actions](https://mlsanity.com/paper/2406.07904/) (2024)
- [Transfer Learning with Synthetic Corpora for Spatial Role Labeling and Reasoning](https://mlsanity.com/paper/2210.16952/) (2022)
- [AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO](https://mlsanity.com/paper/2502.14669/) (2025)
- [Bot or Human? Detecting ChatGPT Imposters with A Single Question](https://mlsanity.com/paper/2305.06424/) (2023)
- [AutoSpatial: Visual-Language Reasoning for Social Robot Navigation through Efficient Spatial Reasoning Learning](https://mlsanity.com/paper/2503.07557/) (2025)
- [Puzzle Solving without Search or Human Knowledge: An Unnatural Language Approach](https://mlsanity.com/paper/2109.02797/) (2021)
- [MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models](https://mlsanity.com/paper/2507.20395/) (2025)
- [Testing the Depth of ChatGPT's Comprehension via Cross-Modal Tasks Based on ASCII-Art: GPT3.5's Abilities in Regard to Recognizing and Generating ASCII-Art Are Not Totally Lacking](https://mlsanity.com/paper/2307.16806/) (2023)
- [Text Speaks Louder than Vision: ASCII Art Reveals Textual Biases in Vision-Language Models](https://mlsanity.com/paper/2504.01589/) (2025)

## Built on by (most cited first, 1 of 1)

- [MemoGuard: An Adaptive Runtime for Guarding Against Memory Traps in Communication-Limited Robot Navigation](https://mlsanity.com/paper/2607.15589/) (2026)

## Abstract (arXiv)

Spatial reasoning in large language models (LLMs) has gained increasing attention due to applications in navigation and planning. Despite strong general language capabilities, LLMs still struggle with spatial transformations and multi-step planning in structured environments. We propose a two-stage approach that decomposes spatial reasoning into atomic building blocks and their composition. First, we apply supervised fine-tuning on elementary spatial transformations, such as rotation, translation, and scaling, to equip the model with basic spatial physics. We then freeze this physics-aware model and train lightweight LoRA adapters within the GRPO framework to learn policies that compose these building blocks for multi-step planning in puzzle-based environments, in a closed-loop manner. To support this pipeline, we synthesize an ASCII-art dataset and construct a corresponding ASCII-based reinforcement learning environment. Our method consistently outperforms baselines, including the generic backbone, physics-aware model, and end-to-end RL models, under both Dynamic environments with explicit state updates and Static environments where the model must rely on its internal state across steps. In addition, the proposed approach converges faster and exhibits more stable training compared to end-to-end reinforcement learning from scratch. Finally, we analyze attention patterns to assess whether fine-tuning induces meaningful improvements in spatial understanding.