mlsanityData to 7 Oct 2026

Geometric Reasoning in the Embedding Space

arXiv 2504.02018 ↗First posted 2 Apr 2025cs.LGcited by 0

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

In this contribution, we demonstrate that Graph Neural Networks and Transformers can learn to reason about geometric constraints. We train them to predict spatial position of points in a discrete 2D grid from a set of constraints that uniquely describe hidden figures containing these points. Both models are able to predict the position of points and interestingly, they form the hidden figures described by the input constraints in the embedding space during the reasoning process. Our analysis shows that both models recover the grid structure during training so that the embeddings corresponding to the points within the grid organize themselves in a 2D subspace and reflect the neighborhood structure of the grid. We also show that the Graph Neural Network we design for the task performs significantly better than the Transformer and is also easier to scale.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 18 of its 25 references that are in the map. First posted on arXiv 2 Apr 2025.

Builds on

18 of 25 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models4 Apr
  2. 2023DepWiGNN: A Depth-wise Graph Neural Network for Multi-hop Spatial Reasoning in Text19 Oct
  3. 2022EuclidNet: Deep Visual Reasoning for Constructible Problems in Geometry27 Dec

Other references · 15

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,841
  5. 2018Universal Transformerscited by 613
  6. 2018Learning a SAT Solver from Single-Bit Supervisioncited by 198
  7. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 118
  8. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  9. 2023Generalization on the Unseen, Logic Reasoning and Degree Curriculumcited by 73
  10. 2023Evaluating Spatial Understanding of Large Language Modelscited by 65
  11. 2017Representation Learning for Grounded Spatial Reasoningcited by 23
  12. 2023Structured World Representations in Maze-Solving Transformerscited by 16
3 more
  1. 2020Automatically Building Diagrams for Olympiad Geometry Problemscited by 8
  2. 2024Understanding GNNs for Boolean Satisfiability through Approximation Algorithmscited by 3
  3. 2025Neural Approaches to SAT Solving: Design Choices and Interpretabilitycited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Fully Geometric Multi-Hop Reasoning on Knowledge Graphs with Transitive Relations
  2. 2026Draw2Think: Harnessing Geometry Reasoning through Constraint Engine Interaction
  3. 2023GeomVerse: A Systematic Evaluation of Large Models for Geometric Reasoning
  4. 2026Hilbert-Geo: Solving Solid Geometric Problems by Neural-Symbolic Reasoning
  5. 2023River of No Return: Graph Percolation Embeddings for Efficient Knowledge Graph Reasoning
  6. 2023Infusing Lattice Symmetry Priors in Attention Mechanisms for Sample-Efficient Abstract Geometric Reasoning
  7. 2025Unraveling the geometry of visual relational reasoning
  8. 2026Geometry without Position? When Positional Embeddings Help and Hurt Spatial Reasoning