# Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence

arXiv: https://arxiv.org/abs/2608.12743 · HTML: https://mlsanity.com/paper/2608.12743/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 22 of its 53 references that are in the map.
- First posted on arXiv 13 Aug 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 22 of 22)

- [Voyager: An Open-Ended Embodied Agent with Large Language Models](https://mlsanity.com/paper/2305.16291/) (2023)
- [MemGPT: Towards LLMs as Operating Systems](https://mlsanity.com/paper/2310.08560/) (2023)
- [RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics](https://mlsanity.com/paper/2411.16537/) (2024)
- [MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory](https://mlsanity.com/paper/2601.03192/) (2026)
- [ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models](https://mlsanity.com/paper/2505.21500/) (2025)
- [SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models](https://mlsanity.com/paper/2412.07755/) (2024)
- [OmniSpatial: Towards Comprehensive Spatial Reasoning Benchmark for Vision Language Models](https://mlsanity.com/paper/2506.03135/) (2025)
- [Gemini Robotics: Bringing AI into the Physical World](https://mlsanity.com/paper/2503.20020/) (2025)
- [SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning](https://mlsanity.com/paper/2606.13673/) (2026)
- [pySpatial: Generating 3D Visual Programs for Zero-Shot Spatial Reasoning](https://mlsanity.com/paper/2603.00905/) (2026)
- [S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence](https://mlsanity.com/paper/2606.20515/) (2026)
- [Memory-R2: Fair Credit Assignment for Long-Horizon Memory-Augmented LLM Agents](https://mlsanity.com/paper/2605.21768/) (2026)
- [AtlasVA: Self-Evolving Visual Skill Memory for Teacher-Free VLM Agents](https://mlsanity.com/paper/2605.17933/) (2026)
- [MemRefine: LLM-Guided Compression for Long-Term Agent Memory](https://mlsanity.com/paper/2606.13177/) (2026)
- [TRUSTMEM: Learning Trustworthy Memory Consolidation for LLM Agents with Long-Term Memory](https://mlsanity.com/paper/2606.25161/) (2026)
- [SpatialEvo: Self-Evolving Spatial Intelligence via Deterministic Geometric Environments](https://mlsanity.com/paper/2604.14144/) (2026)
- [MemQ: Integrating Q-Learning into Self-Evolving Memory Agents over Provenance DAGs](https://mlsanity.com/paper/2605.08374/) (2026)
- [AttriMem: Attribution-Guided Process Feedback for Agent Memory Construction](https://mlsanity.com/paper/2607.21106/) (2026)
- [Self-Evolving Spatial Reasoning in Vision Language Models via Geometric Logic Consistency](https://mlsanity.com/paper/2605.18162/) (2026)
- [memorywire: A Vendor-Neutral Wire Format for Agent Memory Operations](https://mlsanity.com/paper/2606.01138/) (2026)
- [SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them](https://mlsanity.com/paper/2607.27703/) (2026)
- [Ouroboros-Spatial: Closing the Data-Model Loop for Spatial Reasoning](https://mlsanity.com/paper/2606.11719/) (2026)

## Built on by (most cited first, 2 of 2)

- [CoEvoWhen: Policy-Tool Coevolution for Ultra-Long Video Temporal Grounding](https://mlsanity.com/paper/2609.40048/) (2026)
- [Reflect After Failure: Brain-Inspired Self-Evolving Agents for Embodied Spatial Intelligence](https://mlsanity.com/paper/2610.20312/) (2026)

## Abstract (arXiv)

Spatial intelligence is becoming a foundation for embodied agents, robotic planning, and multimodal assistants. To improve the spatial reasoning ability of VLMs, existing work has mainly followed two lines. One line uses post-training methods, such as supervised fine-tuning and reinforcement learning. Another line adopts an agentic paradigm in which the model calls external spatial tools to gather intermediate spatial evidence. We study a complementary and underexplored route: Can a frozen VLM agent improve its spatial reasoning through parameter-update-free self-evolution, without depending on external expert spatial tools at inference time? We present Spatial Memory Agent (SMA), an experience-grounded runtime memory framework that converts verified spatial experience into reusable transferable lessons. Specifically, SMA first queries the frozen VLM in a verifiable spatial environment, obtains a predicted answer and reward, and uses verifier-guided reflection to distill compact transferable lessons stored in memory cards. SMA further assigns each memory card a Transfer Reliability Score (TRS), which is initialized uniformly and calibrated from later retrieval outcomes as visit evidence of future transfer reliability. During read-only deployment, SMA retrieves memory cards through semantic filtering and combined similarity--TRS ranking, allowing the retrieved memory to guide frozen model inference. Experiments across five representative spatial benchmarks show that SMA achieves the best macro-average accuracy for all four base VLMs and the best accuracy in most individual evaluations, establishing a practical parameter-update-free path for spatial self-evolution through reusable experience.