mlsanityData to 7 Oct 2026

AlphaMaze: Enhancing Large Language Models' Spatial Intelligence via GRPO

arXiv 2502.14669 ↗First posted 20 Feb 2025cs.CLcited by 33

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Large Language Models (LLMs) have demonstrated impressive capabilities in language processing, yet they often struggle with tasks requiring genuine visual spatial reasoning. In this paper, we introduce a novel two-stage training framework designed to equip standard LLMs with visual reasoning abilities for maze navigation. First, we leverage Supervised Fine Tuning (SFT) on a curated dataset of tokenized maze representations to teach the model to predict step-by-step movement commands. Next, we apply Group Relative Policy Optimization (GRPO)-a technique used in DeepSeekR1-with a carefully crafted reward function to refine the model's sequential decision-making and encourage emergent chain-of-thought behaviors. Experimental results on synthetically generated mazes show that while a baseline model fails to navigate the maze, the SFT-trained model achieves 86% accuracy, and further GRPO fine-tuning boosts accuracy to 93%. Qualitative analyses reveal that GRPO fosters more robust and self-corrective reasoning, highlighting the potential of our approach to bridge the gap between language models and visual spatial tasks. These findings offer promising implications for applications in robotics, autonomous navigation, and other domains that require integrated visual and sequential reasoning.

Key facts. Cited by 33 papers in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 20 references that are in the map. First posted on arXiv 20 Feb 2025.

Builds on

13 of 20 references.

References · 13

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  6. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  7. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 552
  8. 2024A Survey on Vision-Language-Action Models for Embodied AIcited by 274
  9. 2023Reward Design with Language Modelscited by 257
  10. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 204
  11. 2023A Configurable Library for Generating and Manipulating Maze Datasetscited by 31
  12. 2023Neurosymbolic AI for Reasoning over Knowledge Graphs: A Surveycited by 21
1 more
  1. 2024Supervised Fine-tuning in turn Improves Visual Foundation Modelscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 15.

33 papers.

Sort:

By area:

  1. 2025Reasoning-SQL: Reinforcement Learning with SQL Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQLcited by 59
  2. 2025CAPO: Towards Enhancing LLM Reasoning through Generative Credit Assignmentcited by 25
  3. 2026Large Language Model Reasoning Failurescited by 20
  4. 2026Thinking in Frames: How Visual Context and Test-Time Scaling Empower Video Reasoningcited by 10
  5. 2025RAIDX: A Retrieval-Augmented Generation and GRPO Reinforcement Learning Framework for Explainable Deepfake Detectioncited by 8
  6. 2025MedTVT-R1: A Multimodal LLM Empowering Medical Reasoning and Diagnosiscited by 7
  7. 2025Reasoning LLMs are Wandering Solution Explorerscited by 6
  8. 2025What MLLMs Learn about When they Learn about Multimodal Reasoningcited by 6
  9. 2025A Stitch in Time Saves Nine: Proactive Self-Refinement for Language Modelscited by 4
  10. 2025Less Redundancy: Boosting Practicality of Vision Language Model in Walking Assistantscited by 2
  11. 2025Orchestrator: Active Inference for Multi-Agent Systems in Long-Horizon Taskscited by 2
  12. 2025Constructing coherent spatial memory in LLM agents through graph rectificationcited by 2
21 more
  1. 2025Distilling Counterfactual Reasoning from Language to Vision: Causal Graph Guided Post-Training for Video Understandingcited by 2
  2. 2026STAR: Mitigating Cascading Errors in Spatial Reasoning via Turn-point Alignment and Segment-level DPOcited by 2
  3. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Taskscited by 2
  4. 2026Learning to Correct: Calibrated Reinforcement Learning for Multi-Attempt Chain-of-Thoughtcited by 2
  5. 2026Probing Visual Planning in Image Editing Modelscited by 2
  6. 2026Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Trainingcited by 2
  7. 2025Does RLVR Extend Reasoning Boundaries? Investigating Capability Expansion in Vision-Language Modelscited by 1
  8. 2025The Alignment Paradox of Medical Large Language Models in Infertility Care: Decoupling Algorithmic Improvement from Clinical Decision-making Qualitycited by 1
  9. 2025From Building Blocks to Planning: Multi-Step Spatial Reasoning in LLMs with Reinforcement Learningcited by 1
  10. 2026GraphWalk: Enabling Reasoning in Large Language Models through Tool-Based Graph Navigationcited by 1
  11. 2026Mind the Gap Between Spatial Reasoning and Acting! Step-by-Step Evaluation of Agents With Spatial-Gymcited by 1
  12. 2026Entropy-KL Divergence-based Token Masking: A Novel Approach for Selective Fine-tuning of Large Language Modelscited by 1
  13. 2026Learning Visual Spatial Planning from Symbolic State via Modality-Gap-Aware Self-Distillationcited by 1
  14. 2025AlphaSpace: Enabling Robotic Actions through Semantic Tokenization and Symbolic Reasoningcited by 0
  15. 2025Assessing Consciousness-Related Behaviors in Large Language Models Using the Maze Testcited by 0
  16. 2025Robust Object Detection for Autonomous Driving via Curriculum-Guided Group Relative Policy Optimizationcited by 0
  17. 2025Judging by the Rules: Compliance-Aligned Framework for Modern Slavery Statement Monitoringcited by 0
  18. 2026Deconstructing Spatial Complexity: Hierarchical Decomposition for LLM Spatial Reasoningcited by 0
  19. 2026OmniRouting: A Semantic-Coupled Multimodal Benchmark for Constraint-Aware Spatial Reasoning in PCB Routingcited by 0
  20. 2026MindTopo: Can Foundation Models Reason in Topological Space?cited by 0
  21. 2026The Imitation Game: When LLMs Learn to Reason Like Programs via Code-Centric Reasoning Data Synthesiscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
  2. 2025Enhancing Spatial Reasoning in Vision-Language Models via Chain-of-Thought Prompting and Reinforcement Learning
  3. 2024MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chains
  4. 2026Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design
  5. 2023NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models
  6. 2026ABot-N1: Toward a General Visual Language Navigation Foundation Model
  7. 2025VLN-R1: Vision-Language Navigation via Reinforcement Fine-Tuning
  8. 2025Improved Visual-Spatial Reasoning via R1-Zero-Like Training