mlsanityData to 14 Oct 2026

Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism

arXiv 2511.17198 ↗First posted 21 Nov 2025cs.AI cs.CVcited by 16

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

LLM-driven agents, particularly those using general frameworks like ReAct or human-inspired role-playing, often struggle in specialized domains that necessitate rigorously structured workflows. Fields such as remote sensing, requiring specialized tools (e.g., correction, spectral indices calculation), and multi-step procedures (e.g., numerous intermediate products and optional steps), significantly challenge generalized approaches. To address this gap, we introduce a novel agent design framework centered on a Hierarchical Task Abstraction Mechanism (HTAM). Specifically, HTAM moves beyond emulating social roles, instead structuring multi-agent systems into a logical hierarchy that mirrors the intrinsic task-dependency graph of a given domain. This task-centric architecture thus enforces procedural correctness and decomposes complex problems into sequential layers, where each layer's sub-agents operate on the outputs of the preceding layers. We instantiate this framework as EarthAgent, a multi-agent system tailored for complex geospatial analysis. To evaluate such complex planning capabilities, we build GeoPlan-bench, a comprehensive benchmark of realistic, multi-step geospatial planning tasks. It is accompanied by a suite of carefully designed metrics to evaluate tool selection, path similarity, and logical completeness. Experiments show that EarthAgent substantially outperforms a range of established single- and multi-agent systems. Our work demonstrates that aligning agent architecture with a domain's intrinsic task structure is a critical step toward building robust and reliable specialized autonomous systems.

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 33 of its 68 references that are in the map. First posted on arXiv 21 Nov 2025.

Builds on

33 of 68 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents27 Sep
  2. 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks29 May
  3. 2025Multi-Agent Geospatial Copilots for Remote Sensing Workflows27 Jan
  4. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent11 Jun

Other references · 29

  1. 2025Qwen3 Technical Reportcited by 8,820
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  6. 2023A Survey of Large Language Modelscited by 2,576
  7. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,212
  8. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  9. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  10. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  11. 2023ChatDev: Communicative Agents for Software Developmentcited by 919
  12. 2023ChemCrow: Augmenting large-language models with chemistry toolscited by 631
17 more
  1. 2023Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Modelscited by 560
  2. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  3. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 295
  4. 2024Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judgecited by 269
  5. 2024LAB-Bench: Measuring Capabilities of Language Models for Biology Researchcited by 156
  6. 2025Beyond Self-Talk: A Communication-Centric Survey of LLM-Based Multi-Agent Systemscited by 62
  7. 2024HyperAgent: Generalist Software Engineering Agents to Solve Coding Tasks at Scalecited by 61
  8. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Taskscited by 47
  9. 2025SegEarth-R1: Geospatial Pixel Reasoning via Large Language Modelcited by 42
  10. 2023LLF-Bench: Benchmark for Interactive Learning from Language Feedbackcited by 33
  11. 2025From Mind to Machine: The Rise of Manus AI as a Fully Autonomous Digital Agentcited by 33
  12. 2023Automating the Generation of Prompts for LLM-based Action Choice in PDDL Planningcited by 24
  13. 2025DynamicEarth: How Far are We from Open-Vocabulary Change Detection?cited by 21
  14. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
  15. 2025Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Imagescited by 8
  16. 2024Reflection-Bench: Evaluating Epistemic Agency in Large Language Modelscited by 7
  17. 2025DescribeEarth: Describe Anything for Remote Sensing Imagescited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 15.

16 papers.

Sort:

By area:

  1. 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observationcited by 9
  2. 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applicationscited by 8
  3. 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMscited by 8
  4. 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysiscited by 5
  5. 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observerscited by 4
  6. 2026RSMeM: Knowledge-Enhanced Memory Evolution for Remote Sensing Agents with Systematic Evaluationcited by 2
  7. 2026Bidirectional Semantic Complementary Tool Retrieval for Remote Sensing Agentscited by 1
  8. 2026GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoningcited by 1
  9. 2026HiRS-Agent: A Hierarchical Multi-Agent System for Reliable Long-Horizon Remote Sensing Task Solvingcited by 1
  10. 2026Think with Extra-Image: A Farmland Segmentation Agent Driven by Spatio-Temporal Information Gaincited by 0
  11. 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuningcited by 0
  12. 2026Deploying and Evaluating a Smart-Agriculture Agentic Engine for Full-Season Soybean Farm Operationscited by 0
4 more
  1. 2026STQA: A Benchmark for Stock-Focused Tabular Question Answering over Historical and Forecasted Datacited by 0
  2. 2026Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agentscited by 0
  3. 2026GeoSkill:Experience-Driven Hierarchical Skill Learning with Collaborative Revision forGeospatialAgentscited by 0
  4. 2026WeaveAgent: A Two-Stage Tool-Routing Agent for Ultra-High-Resolution Remote Sensing Imagerycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025MapAgent: A Hierarchical Agent for Geospatial Reasoning with Dynamic Map Tool Integration
  2. 2026TerraLogic: A Benchmark for Hierarchical Geospatial Reasoning in Earth Observation
  3. 2026Bridging Perception and Action: A Lightweight Multimodal Meta-Planner Framework for Robust Earth Observation Agents ↗
  4. 2026Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems
  5. 2025A survey of multi-agent geosimulation methodologies: from ABM to LLM
  6. 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agents
  7. 2026GeoNatureAgent (GNA): A Framework and Benchmark for Pre-Production Evaluation of Tool-Using Agents on Geospatial and Environmental Tasks
  8. 2026Spatial-Agent: Agentic Geo-spatial Reasoning with Scientific Core Concepts