# S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence

arXiv: https://arxiv.org/abs/2606.20515 · HTML: https://mlsanity.com/paper/2606.20515/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 29 of its 33 references that are in the map.
- First posted on arXiv 18 Jun 2026.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 29 of 29)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [VGGT: Visual Geometry Grounded Transformer](https://mlsanity.com/paper/2503.11651/) (2025)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://mlsanity.com/paper/2403.13372/) (2024)
- [Depth Anything 3: Recovering the Visual Space from Any Views](https://mlsanity.com/paper/2511.10647/) (2025)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces](https://mlsanity.com/paper/2412.14171/) (2024)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)
- [MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Action](https://mlsanity.com/paper/2303.11381/) (2023)
- [Long Context Transfer from Language to Vision](https://mlsanity.com/paper/2406.16852/) (2024)
- [Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence](https://mlsanity.com/paper/2505.23747/) (2025)
- [MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligence](https://mlsanity.com/paper/2505.23764/) (2025)
- [SpaceR: Reinforcing MLLMs in Video Spatial Reasoning](https://mlsanity.com/paper/2504.01805/) (2025)
- [3DSRBench: A Comprehensive 3D Spatial Reasoning Benchmark](https://mlsanity.com/paper/2412.07825/) (2024)
- [Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing](https://mlsanity.com/paper/2506.09965/) (2025)
- [Cambrian-S: Towards Spatial Supersensing in Video](https://mlsanity.com/paper/2511.04670/) (2025)
- [ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Models](https://mlsanity.com/paper/2505.21500/) (2025)
- [MindCube: Spatial Mental Modeling from Limited Views](https://mlsanity.com/paper/2506.21458/) (2025)
- [T*: Re-thinking Temporal Search for Long-Form Video Understanding](https://mlsanity.com/paper/2504.02259/) (2025)
- [Visual Agentic AI for Spatial Reasoning with a Dynamic API](https://mlsanity.com/paper/2502.06787/) (2025)
- [Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcuts](https://mlsanity.com/paper/2511.04655/) (2025)
- [Geometrically-Constrained Agent for Spatial Reasoning](https://mlsanity.com/paper/2511.22659/) (2025)
- [Think3D: Thinking with Space for Spatial Reasoning](https://mlsanity.com/paper/2601.13029/) (2026)
- [SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RL](https://mlsanity.com/paper/2512.04069/) (2025)
- [ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoning](https://mlsanity.com/paper/2604.24300/) (2026)
- [Solving Spatial Supersensing Without Spatial Supersensing](https://mlsanity.com/paper/2511.16655/) (2025)

## Built on by (most cited first, 11 of 11)

- [SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them](https://mlsanity.com/paper/2607.27703/) (2026)
- [Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence](https://mlsanity.com/paper/2608.12743/) (2026)
- [R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables](https://mlsanity.com/paper/2607.02921/) (2026)
- [CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environments](https://mlsanity.com/paper/2609.32427/) (2026)
- [Uruqi: Learning Spatial Cognition from Visual Experience](https://mlsanity.com/paper/2609.39195/) (2026)
- [From Reasoning Failures to Composable Video Spatial Intelligence](https://mlsanity.com/paper/2610.01999/) (2026)
- [SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces](https://mlsanity.com/paper/2610.11366/) (2026)
- [SpatialASM: Internalizing Active Mentalization for Spatial Reasoning](https://mlsanity.com/paper/2610.13832/) (2026)
- [HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoning](https://mlsanity.com/paper/2610.14679/) (2026)
- [ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoning](https://mlsanity.com/paper/2610.16315/) (2026)
- [MemAct: Streaming Spatial Understanding with Memory and Active Exploration](https://mlsanity.com/paper/2610.17640/) (2026)

## Abstract (arXiv)

Real-world spatial intelligence requires reasoning over a continuous and evolving 3D world, yet existing VLMs and tool-augmented agents largely remain tied to static, stateless inference from isolated visual observations. We introduce textbftextscS-Agent, a spatial tool-use agentic paradigm for understanding and reasoning over continuous multi-view images and videos. By formulating spatial reasoning as spatio-temporal evidence accumulation rather than isolated frame-level prediction, textscS-Agent reshapes spatial perception into scene-centric understanding beyond frame-centric recognition. Specifically, textscS-Agent casts the VLM as a semantic planner that decides what evidence is needed, while a hierarchy of spatial tools and experts grounds objects in 2D, lifts them into 3D geometric evidence, and aggregates this evidence into high-level spatial knowledge (textite.g., counting, measurement, orientation, and relative position). Additionally, a temporal memory mechanism, including Scene Memory for maintaining the evolving scene state and Agent Memory for accumulating reasoning context, enables evidence integration across frames and reasoning steps. Comprehensive experiments on multi-view and video spatial reasoning benchmarks show that textscS-Agent consistently improves both open-source and closed-source VLMs in a training-free manner. Beyond inference-time augmentation, supervised fine-tuning (SFT) on textscS-Agent-generated spatial trajectories textscS-300K yields textscS-Agent-8B, a compact spatial agent that significantly surpasses similar-scale baselines (e.g., Qwen3-VL-8B) and performs comparably to advanced closed-source models (e.g., GPT-5.4 and Gemini 3).