Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
LLM-driven agents, particularly those using general frameworks like ReAct or human-inspired role-playing, often struggle in specialized domains that necessitate rigorously structured workflows. Fields such as remote sensing, requiring specialized tools (e.g., correction, spectral indices calculation), and multi-step procedures (e.g., numerous intermediate products and optional steps), significantly challenge generalized approaches. To address this gap, we introduce a novel agent design framework centered on a Hierarchical Task Abstraction Mechanism (HTAM). Specifically, HTAM moves beyond emulating social roles, instead structuring multi-agent systems into a logical hierarchy that mirrors the intrinsic task-dependency graph of a given domain. This task-centric architecture thus enforces procedural correctness and decomposes complex problems into sequential layers, where each layer's sub-agents operate on the outputs of the preceding layers. We instantiate this framework as EarthAgent, a multi-agent system tailored for complex geospatial analysis. To evaluate such complex planning capabilities, we build GeoPlan-bench, a comprehensive benchmark of realistic, multi-step geospatial planning tasks. It is accompanied by a suite of carefully designed metrics to evaluate tool selection, path similarity, and logical completeness. Experiments show that EarthAgent substantially outperforms a range of established single- and multi-agent systems. Our work demonstrates that aligning agent architecture with a domain's intrinsic task structure is a critical step toward building robust and reliable specialized autonomous systems.
Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 68 references that are in the map. First posted on arXiv 21 Nov 2025.
Builds on
33 of 68 references.
Closest ancestors · 4
Same direction, newest first
- 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents27 Sep
- 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks29 May
- 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflows27 Jan
- 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent11 Jun
Other references · 29
- 2025Qwen3 Technical Reportcited by 8,820
- 2024GPT-4o System Cardcited by 4,588
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023A Survey of Large Language Modelscited by 2,576
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
- 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
- 2023ChatDev: Communicative Agents for Software Developmentcited by 919
- 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
17 more
- 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
- 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
- 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
- 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
- 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
- 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 62
- 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
- 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 47
- 2025SegEarth-R1: Geospatial Pixel Reasoning via Large Language Modelcited by 42
- 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 33
- 2025From Mind to Machine: The Rise of Manus AI as a Fully Autonomous Digital Agentcited by 33
- 2023Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planningcited by 24
- 2025DynamicEarth: How Far are We from Open-Vocabulary Change Detection?cited by 21
- 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
- 2025Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Imagescited by 8
- 2024Reflection-Bench: Evaluating Epistemic Agency in Large Language Modelscited by 7
- 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 15.
16 papers.
Sort:
By area:
- 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observationcited by 9
- 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applicationscited by 8
- 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMscited by 8
- 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysiscited by 5
- 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observerscited by 4
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluationcited by 2
- 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agentscited by 1
- 2026GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoningcited by 1
- 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solvingcited by 1
- 2026Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gaincited by 0
- 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuningcited by 0
- 2026Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operationscited by 0
4 more
- 2026STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Datacited by 0
- 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agentscited by 0
- 2026GeoSkill:Experience-Driven Hierarchical Skill Learning with Collaborative Revision forGeospatialAgentscited by 0
- 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagerycited by 0
- 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagery25 Sep 2026
- 2026GeoSkill:Experience-Driven Hierarchical Skill Learning with Collaborative Revision forGeospatialAgents12 Sep 2026
- 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents11 Sep 2026
- 2026STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data5 Sep 2026
- 2026Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operations31 Aug 2026
- 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving31 Aug 2026
- 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning31 Aug 2026
- 2026GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoning11 Aug 2026
- 2026Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gain30 Jul 2026
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation11 Jun 2026
- 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents29 Apr 2026
- 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis15 Apr 2026
4 more
- 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs9 Apr 2026
- 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation23 Mar 2026
- 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observers30 Jan 2026
- 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications17 Dec 2025
- 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observationcited by 9
- 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applicationscited by 8
- 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMscited by 8
- 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysiscited by 5
- 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observerscited by 4
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluationcited by 2
- 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agentscited by 1
- 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solvingcited by 1
- 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuningcited by 0
- 2026Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operationscited by 0
- 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agentscited by 0
- 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents11 Sep 2026
- 2026Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operations31 Aug 2026
- 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solving31 Aug 2026
- 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning31 Aug 2026
- 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluation11 Jun 2026
- 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agents29 Apr 2026
- 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis15 Apr 2026
- 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs9 Apr 2026
- 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation23 Mar 2026
- 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observers30 Jan 2026
- 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications17 Dec 2025
- 2026Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gaincited by 0
- 2026STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Datacited by 0
- 2026GeoSkill:Experience-Driven Hierarchical Skill Learning with Collaborative Revision forGeospatialAgentscited by 0
- 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagerycited by 0
- 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagery25 Sep 2026
- 2026GeoSkill:Experience-Driven Hierarchical Skill Learning with Collaborative Revision forGeospatialAgents12 Sep 2026
- 2026STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Data5 Sep 2026
- 2026Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gain30 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2025MapAgent: A Hierarchical Agent for Geospatial Reasoning with Dynamic Map Tool Integration
- 2026TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation
- 2026Bridging Perception and Action: A Lightweight Multimodal Meta-Planner Framework for Robust Earth Observation Agents ↗
- 2026Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems
- 2025A survey of multi-agent geosimulation methodologies: from ABM to LLM
- 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents
- 2026GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks
- 2026Spatial-Agent: Agentic Geo-spatial Reasoning with Scientific Core Concepts