mlsanityData to 7 Oct 2026

Interaction Locality in Hierarchical Recursive Reasoning

arXiv 2605.20784 ↗First posted 20 May 2026cs.AI cs.LGcited by 1

On the map: LLM reasoning → Spatial and embodied reasoning

Abstract (arXiv)

Spatial reasoning requires both location-bound computation and location-invariant structure: agents must make local moves while preserving route, object, or constraint-level plans. We propose interaction locality, a task-geometry-aware framework for measuring whether information flow stays within nearby cells or semantic segments, or crosses them. We instantiate the framework with sparse-autoencoder feature ablations and finite-noise activation patching, with structural Jacobian and attention checks reported in the appendix, and apply it to HRM and TRM, two compact hierarchical and recursive reasoning models, on Maze-Hard, Sudoku Extreme, and ARC-AGI. Across these models, activation patching gives the clearest architectural fingerprint: high-level recurrent states tend to write information within nearby cells or same-segment units, while repeated recursive updates accumulate these local writes into broader solution structure. This pattern holds across maze paths, Sudoku constraints, and ARC-AGI object neighborhoods, with the strongest concentration in TRM. To test whether interaction locality extends beyond toy-yet-challenging grid benchmarks, we also apply it to MTU3D, a large-scale embodied 3D scene-grounding model. In this MTU3D setting, causal spatial locality appears primarily at the transition where visual scene features are handed to the downstream grounding module, rather than uniformly throughout the visual encoder. This contrast suggests that the local-to-global handoff observed in HRM and TRM is tied to explicit recursive reasoning dynamics, while embodied 3D models may concentrate causal spatial structure at module boundaries. Interaction locality turns the intuitive local-execution/global-planning story into a reproducible measurement framework for recursive and embodied spatial reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Spatial and embodied reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 20 references that are in the map. First posted on arXiv 20 May 2026.

Builds on

17 of 20 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models18 Jan
  2. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing11 Jun
  3. 2024SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities22 Jan

Other references · 14

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  3. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  4. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  5. 2016Adaptive Computation Time for Recurrent Neural Networkscited by 479
  6. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  7. 2025Hierarchical Reasoning Modelcited by 167
  8. 2023Towards Learning a Generalist Model for Embodied Navigationcited by 135
  9. 2021Chess as a Testbed for Language Model State Trackingcited by 69
  10. 2025Move to Understand a 3D Scene: Bridging Visual Grounding and Exploration for Efficient and Versatile Embodied Navigationcited by 50
  11. 2024Evidence of Learned Look-Ahead in a Chess-Playing Neural Networkcited by 33
  12. 2023Structured World Representations in Maze-Solving Transformerscited by 16
2 more
  1. 2024Transformers Use Causal World Models in Maze-Solving Taskscited by 13
  2. 2025Geospatial Mechanistic Interpretability of Large Language Modelscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Spatial-Interactor: Learning Spatial Reasoning through Interaction with the Observable Physical World
  2. 2026SpaMEM: Benchmarking Dynamic Spatial Reasoning via Perception-Memory Integration in Embodied Environments
  3. 2025MazeEval: A Benchmark for Testing Sequential Decision-Making in Language Models
  4. 2026Do LLMs Build Spatial World Models? Evidence from Grid-World Maze Tasks
  5. 2026Skill-3D: Evolving Scene-Aware Skills for Agentic 3D Spatial Reasoning
  6. 2025Stuck in the Matrix: Probing Spatial Reasoning in Large Language Models
  7. 2025EmbodiedVSR: Dynamic Scene Graph-Guided Chain-of-Thought Reasoning for Visual Spatial Tasks
  8. 2025REM: Evaluating LLM Embodied Spatial Reasoning through Multi-Frame Trajectories