S-Agent: Spatial Tool-Use Elicits Reasoning for Spatial Intelligence
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Real-world spatial intelligence requires reasoning over a continuous and evolving 3D world, yet existing VLMs and tool-augmented agents largely remain tied to static, stateless inference from isolated visual observations. We introduce textbftextscS-Agent, a spatial tool-use agentic paradigm for understanding and reasoning over continuous multi-view images and videos. By formulating spatial reasoning as spatio-temporal evidence accumulation rather than isolated frame-level prediction, textscS-Agent reshapes spatial perception into scene-centric understanding beyond frame-centric recognition. Specifically, textscS-Agent casts the VLM as a semantic planner that decides what evidence is needed, while a hierarchy of spatial tools and experts grounds objects in 2D, lifts them into 3D geometric evidence, and aggregates this evidence into high-level spatial knowledge (textite.g., counting, measurement, orientation, and relative position). Additionally, a temporal memory mechanism, including Scene Memory for maintaining the evolving scene state and Agent Memory for accumulating reasoning context, enables evidence integration across frames and reasoning steps. Comprehensive experiments on multi-view and video spatial reasoning benchmarks show that textscS-Agent consistently improves both open-source and closed-source VLMs in a training-free manner. Beyond inference-time augmentation, supervised fine-tuning (SFT) on textscS-Agent-generated spatial trajectories textscS-300K yields textscS-Agent-8B, a compact spatial agent that significantly surpasses similar-scale baselines (e.g., Qwen3-VL-8B) and performs comparably to advanced closed-source models (e.g., GPT-5.4 and Gemini 3).
Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 29 of its 33 references that are in the map. First posted on arXiv 18 Jun 2026.
Builds on
29 of 33 references.
Closest ancestors · 2
Same direction, newest first
- 2025Geometrically-Constrained Agent for Spatial Reasoning27 Nov
- 2025Visual Agentic AI for Spatial Reasoning with a Dynamic API10 Feb
Other references · 27
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2019nuScenes: A multimodal dataset for autonomous drivingcited by 4,523
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
- 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 611
- 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 550
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
15 more
- 2024Long Context Transfer from Language to Visioncited by 515
- 2025Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligencecited by 217
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
- 2025SpaceR: Reinforcing MLLMs in Video Spatial Reasoningcited by 159
- 20243DSRBench: A Comprehensive 3D Spatial Reasoning Benchmarkcited by 139
- 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawingcited by 136
- 2025Cambrian-S: Towards Spatial Supersensing in Videocited by 134
- 2025ViewSpatial-Bench: Evaluating Multi-perspective Spatial Localization in Vision-Language Modelscited by 118
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 95
- 2025T*: Re-thinking Temporal Search for Long-Form Video Understandingcited by 82
- 2025Benchmark Designers Should "Train on the Test Set" to Expose Exploitable Non-Visual Shortcutscited by 29
- 2026Think3D: Thinking with Space for Spatial Reasoningcited by 24
- 2025SpaceTools: Tool-Augmented Spatial Reasoning via Double Interactive RLcited by 21
- 2026ReVSI: Rebuilding Visual Spatial Intelligence Evaluation for Accurate Assessment of VLM 3D Reasoningcited by 15
- 2025Solving Spatial Supersensing Without Spatial Supersensingcited by 2
Built on by
Citing papers per year: 2026: 11.
11 papers.
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Themcited by 2
- 2026Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligencecited by 2
- 2026R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearablescited by 0
- 2026CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environmentscited by 0
- 2026Uruqi: Learning Spatial Cognition from Visual Experiencecited by 0
- 2026From Reasoning Failures to Composable Video Spatial Intelligencecited by 0
- 2026SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Tracescited by 0
- 2026SpatialASM: Internalizing Active Mentalization for Spatial Reasoningcited by 0
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoningcited by 0
- 2026ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoningcited by 0
- 2026MemAct: Streaming Spatial Understanding with Memory and Active Explorationcited by 0
- 2026MemAct: Streaming Spatial Understanding with Memory and Active Exploration13 Oct 2026
- 2026ReTAC: Residual Test-Time Agentic Computation for Video Spatial Reasoning12 Oct 2026
- 2026HumanPhys: A Human-Centric Physical-World Perception Benchmark for Real-World Visual Reasoning10 Oct 2026
- 2026SpatialASM: Internalizing Active Mentalization for Spatial Reasoning9 Oct 2026
- 2026SpatialOPSD: Self-Distilling Spatial Intelligence from Verified Coding Agent Traces8 Oct 2026
- 2026From Reasoning Failures to Composable Video Spatial Intelligence1 Oct 2026
- 2026Uruqi: Learning Spatial Cognition from Visual Experience30 Sep 2026
- 2026CityToolVQA: Tool-Augmented Visual Question Answering for 3D Spatial Cognition in Urban Low-Altitude Environments26 Sep 2026
- 2026Spatial Memory Agent: Experience-Grounded Procedure Memory for Spatial Intelligence13 Aug 2026
- 2026SpatialCLI: Learning to Reason With Spatial Tools, Then Without Them30 Jul 2026
- 2026R3D: Quantitative 3D Spatial Reasoning for Egocentric Wearables3 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Skill-3D: Evolving Scene-Aware Skills for Agentic 3D Spatial Reasoning
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning
- 2026SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning
- 2026SpatiO: Adaptive Test-Time Orchestration of Vision-Language Agents for Spatial Reasoning
- 2026From Perception to Action: Spatial AI Agents and World Models
- 2025Reasoning Path and Latent State Analysis for Multi-view Visual Spatial Reasoning: A Cognitive Science Perspective
- 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenes
- 2026DeSpatial: The devil of Agentic Spatial Reasoning Lies in 3D Itself