RS-Claw: Progressive Active Tool Exploration via Hierarchical Skill Trees for Remote Sensing Agents
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
The rise of multi-modal large language models (MLLMs) is shifting remote sensing (RS) intelligence from "see" to "action", as OpenClaw-style frameworks enable agents to autonomously operate massive RS image-processing tools for complex tasks. Existing RS agents adopt a passive selection paradigm for tool invocation, relying on either full tool registration (Flat) or retrieval-augmented generation (RAG). However, in the massive and multi-source heterogeneous RS tool ecosystem, such passive mechanisms struggle to dynamically balance "context load" and "toolset completeness" throughout task reasoning, thus exhibiting inherent limitations: full tool registration triggers context space deficits during long-horizon tasks, whereas RAG retrieval may omit critical tools in essential steps. To overcome these bottlenecks, this paper redefines tool selection by arguing that the agent should act as an active explorer within the tool space. Based on this perspective, we propose RS-Claw, a novel RS agent architecture. By leveraging Skill encapsulation technology at the tool end, this architecture hierarchically structures tool descriptions, enabling the agent to execute on-demand sequential decision-making: initially selecting relevant skill branches by reading only tool summaries, then dynamically loading detailed descriptions, and ultimately achieving precise invocation. This active paradigm not only significantly liberates the agent's context space but also effectively ensures the accurate hit rate of critical tools during long-horizon reasoning. Systematic experiments on the Earth-Bench benchmark demonstrate that RS-Claw's active exploration mechanism effectively filters semantic noise and substantially frees up reasoning space, achieving an input token compression ratio of up to 86%, and comprehensively outperforming existing Flat and RAG baselines across complex reasoning evaluations.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 61 references that are in the map. First posted on arXiv 13 May 2026.
Builds on
37 of 61 references.
Closest ancestors · 6
Same direction, newest first
- 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialogues19 Dec
- 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent11 Jun
- 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domain30 Jan
- 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensing24 Nov
- 2023RSGPT: A Remote Sensing Vision Language Model and Benchmark28 Jul
- 2023RemoteCLIP: A Vision Language Foundation Model for Remote Sensing19 Jun
Other references · 31
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
- 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
- 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2023LongBench: A Bilingual, Multitask Benchmark for Long Context Understandingcited by 1,417
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
19 more
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,190
- 2023API-Bank: A Comprehensive Benchmark for Tool-Augmented LLMscited by 591
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
- 2024Tool Learning with Large Language Models: A Surveycited by 223
- 2023SkySense: A Multi-Modal Remote Sensing Foundation Model Towards Universal Interpretation for Earth Observation Imagerycited by 205
- 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
- 2024AnyTool: Self-Reflective, Hierarchical Agents for Large-Scale API Callscited by 124
- 2025The Tool Decathlon: Benchmarking Language Agents for Diverse, Realistic, and Long-Horizon Task Executioncited by 85
- 2024ToolGen: Unified Tool Retrieval and Calling via Generationcited by 74
- 2024Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysiscited by 63
- 2024ToolNet: Connecting Large Language Models with Massive Tools via Tool Graphcited by 60
- 2026Graph-of-Skills: Dependency-Aware Structural Retrieval for Massive Agent Skillscited by 55
- 2026SkillNet: Create, Evaluate, and Connect AI Skillscited by 53
- 2024Re-Invoke: Tool Invocation Rewriting for Zero-Shot Tool Retrievalcited by 52
- 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Taskscited by 46
- 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agentscited by 46
- 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflowscited by 22
- 2024Evaluating Tool-Augmented Agents in Remote Sensing Platformscited by 17
- 2025GraphCogent: Mitigating LLMs' Working Memory Constraints via Multi-Agent Collaboration in Complex Graph Understandingcited by 9
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?cited by 0
- 2026RS-Claw-Evolution: Environment-Feedback-Driven Evolution for Lightweight Remote Sensing Agents in Long-Horizon Taskscited by 0
- 2026RSure-Agent: Reliable Use of Tool Observations for Remote Sensing Agentscited by 0
- 2026RSure-Agent: Reliable Use of Tool Observations for Remote Sensing Agents4 Oct 2026
- 2026RS-Claw-Evolution: Environment-Feedback-Driven Evolution for Lightweight Remote Sensing Agents in Long-Horizon Tasks6 Sep 2026
- 2026Multimodal Large Language Models for Remote Sensing Image Understanding: Domain-Specific or General-Purpose?22 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents
- 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving
- 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents
- 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation
- 2026Bridging Perception and Action: A Lightweight Multimodal Meta-Planner Framework for Robust Earth Observation Agents ↗
- 2026Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems
- 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation