mlsanityData to 14 Oct 2026

Earth-Agent-Pro: Towards Real-World Full-Chain Earth Observation with Agents

arXiv 2609.12533 ↗First posted 11 Sep 2026cs.CV cs.AI cs.CLcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Real-world Earth observation (EO) agents must translate high-level scientific questions into executable workflows to acquire observations, prepare data, perform domain computations, and derive conclusions from runtime evidence. Existing EO agents typically start from supplied observations, while benchmarks typically provide prepared inputs or candidate answers, leaving full-chain open-world EO execution largely untested. We present Earth-Agent-Pro, an execution-adaptive Plan-and-Execute framework using expert-authored skills to constrain planning and runtime tool use. Workflow-centered structured memory records planned steps, accepted evidence, and their dependencies, enabling repair of only the affected workflow suffix when runtime evidence invalidates a step. Separate large language model adapters use sequence-level supervised fine-tuning for planner workflow composition and node-level group relative policy optimization with locally verifiable rewards for executor tool-argument grounding. Earth-Bench-Pro instantiates 248 expert-curated task cores as 744 questions under three matched regimes. Its 248 Open-World Execution questions span RGB imagery, spectral observations, and remote sensing products, pairing high-level requests with runtime data requirements, executable trajectories, and open-ended answers grounded in execution evidence. With a shared GPT-5 backbone, Earth-Agent-Pro achieves 66.13% LLM-as-Judge accuracy, exceeding ReAct by 20.95 points in this metric and 24.44 points in Tools-In-Order. Joint adapter tuning raises Qwen3.5-9B LLM-as-Judge accuracy from 38.31% to 50.00%, an 11.69-point gain over the untuned configuration. Planning-only evaluation and execution with the reference workflow show that the adapters improve workflow composition and argument grounding, respectively. Code and datasets will be released soon.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 70 of its 86 references that are in the map. First posted on arXiv 11 Sep 2026.

Builds on

70 of 86 references.

Closest ancestors · 14

Same direction, newest first

  1. 2026Bringing Agentic Search to Earth Observation Data Discovery2 Jul
  2. 2026TerraBench: Can Agents Reason Over Heterogeneous Earth-System Data?11 Jun
  3. 2026Can LLM Agents Respond to Disasters? Benchmarking Heterogeneous Geospatial Reasoning in Emergency Operations12 May
  4. 2026GeoAgentBench: A Dynamic Execution Benchmark for Tool-Augmented Agents in Spatial Analysis15 Apr
  5. 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs9 Apr
  6. 2026OpenEarth-Agent: From Tool Calling to Tool Creation for Open-Environment Earth Observation23 Mar
  7. 2026Experience-Driven Multi-Agent Systems Are Training-free Context-aware Earth Observers30 Jan
  8. 2025CangLing-KnowFlow: A Unified Knowledge-and-Flow-fused Agent for Comprehensive Remote Sensing Applications17 Dec
  9. 2025Designing Domain-Specific Agents via Hierarchical Task Abstraction Mechanism21 Nov
  10. 2025Earth-Agent: Unlocking the Full Landscape of Earth Observation with Agents27 Sep
  11. 2025ThinkGeo: Evaluating Tool-Augmented Agents for Remote Sensing Tasks29 May
  12. 2025GeoBenchX: Benchmarking LLMs in Agent Solving Multistep Geospatial Tasks23 Mar
2 more
  1. 2025Towards a Barrier-free GeoQA Portal: Natural Language Interaction with Geospatial Data Using Multi-Agent LLMs and Semantic Search18 Mar
  2. 2024RS-Agent: Automating Remote Sensing Tasks through Intelligent Agent11 Jun

Other references · 56

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  4. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,947
  6. 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,332
  7. 2024AFlow: Automating Agentic Workflow Generationcited by 432
  8. 2023GeoChat: Grounded Large Vision-Language Model for Remote Sensingcited by 328
  9. 2023SpectralGPT: Spectral Remote Sensing Foundation Modelcited by 252
  10. 2023RSGPT: A Remote Sensing Vision Language Model and Benchmarkcited by 207
  11. 2020RSVQA: Visual Question Answering for Remote Sensing Datacited by 203
  12. 2024EarthGPT: A Universal Multi-modal Large Language Model for Multi-sensor Image Comprehension in Remote Sensing Domaincited by 184
38 more
  1. 2024LHRS-Bot: Empowering Remote Sensing with VGI-Enhanced Large Multimodal Language Modelcited by 138
  2. 2025AlphaEarth Foundations: An embedding field model for accurate and efficient global mapping from sparse label datacited by 137
  3. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 121
  4. 2024SkyEyeGPT: Unifying Remote Sensing Vision-Language Tasks via Instruction Tuning with Large Language Modelcited by 113
  5. 2024SkySenseGPT: A Fine-Grained Instruction Tuning Dataset and Model for Remote Sensing Vision-Language Understandingcited by 104
  6. 2024VHM: Versatile and Honest Vision Language Model for Remote Sensing Image Analysiscited by 86
  7. 2024EarthDial: Turning Multi-sensory Earth Observations to Interactive Dialoguescited by 85
  8. 2024HyperSIGMA: Hyperspectral Intelligence Comprehension Foundation Modelcited by 73
  9. 2024TEOChat: A Large Vision-Language Assistant for Temporal Earth Observation Datacited by 70
  10. 2024Change-Agent: Towards Interactive Comprehensive Remote Sensing Change Interpretation and Analysiscited by 63
  11. 2025GeoPixel: Pixel Grounding Large Multimodal Model in Remote Sensingcited by 57
  12. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 51
  13. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 51
  14. 2024Cross-view image geo-localization with Panorama-BEV Co-Retrieval Networkcited by 40
  15. 2024Good at captioning, bad at counting: Benchmarking GPT-4V on Earth observation datacited by 39
  16. 2024Where am I? Cross-View Geo-localization with Natural Language Descriptionscited by 38
  17. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 36
  18. 2024Remote Sensing ChatGPT: Solving Remote Sensing Tasks with ChatGPT and Visual Modelscited by 32
  19. 2024GeoLLM-Engine: A Realistic Environment for Building Geospatial Copilotscited by 30
  20. 2024EarthMarker: A Visual Prompting Multi-modal Large Language Model for Remote Sensingcited by 29
  21. 2024UrBench: A Comprehensive Benchmark for Evaluating Large Multimodal Models in Multi-View Urban Scenarioscited by 28
  22. 2024CrossEarth: Geospatial Vision Foundation Model for Domain Generalizable Remote Sensing Semantic Segmentationcited by 27
  23. 2024CrossViewDiff: A Cross-View Diffusion Model for Satellite-to-Street View Synthesiscited by 23
  24. 2024CHOICE: Benchmarking the Remote Sensing Capabilities of Large Vision-Language Modelscited by 22
  25. 2022OmniCity: Omnipotent City Understanding with Multi-level and Multi-view Imagescited by 21
  26. 2025RingMo-Agent: A Unified Remote Sensing Foundation Model for Multi-Platform and Multi-Modal Reasoningcited by 20
  27. 2024SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentationcited by 18
  28. 2025Towards LLM Agents for Earth Observationcited by 17
  29. 2025RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretationcited by 16
  30. 2024RingMo-Aerial: An Aerial Remote Sensing Foundation Model With Affine Transformation Contrastive Learningcited by 15
  31. 2026OpenEarthAgent: A Unified Framework for Tool-Augmented Geospatial Agentscited by 15
  32. 2025OmniEarth-Bench: Towards Holistic Evaluation of Earth's Six Spheres and Cross-Spheres Interactions with Multimodal Observational Earth Datacited by 14
  33. 20243D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisionscited by 13
  34. 2024A2-MAE: A spatial-temporal-spectral unified remote sensing pre-training method based on anchor-aware masked autoencodercited by 11
  35. 2024Leveraging BEV Paradigm for Ground-to-Aerial Image Synthesiscited by 11
  36. 2024RescueADI: Adaptive Disaster Interpretation in Remote Sensing Images with Autonomous Agentscited by 11
  37. 2025EarthGPT-X: A Spatial MLLM for Multi-level Multi-Source Remote Sensing Imagery Understanding with Visual Promptingcited by 11
  38. 2025Can Large Multimodal Models Understand Agricultural Scenes? Benchmarking with AgroMindcited by 8

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GeoForge: Non-Parametric Self-Evolving Agents for Earth-Observation Reasoning
  2. 2026Agentic AI for Remote Sensing: Technical Challenges and Research Directions
  3. 2026SimCRAFT: Distilling Remote Sensing Agents via Synthetic Trajectories and Contextual Retrieval-Augmented Fine-Tuning
  4. 2025Geo-OLM: Enabling Sustainable Earth Observation Studies with Cost-Efficient Open Language Models & State-Driven Workflows
  5. 2026The Last Remote Sensing Model Built by Humans? Casting Any Earth Observation Task as Recursive Self-Improvement
  6. 2026IC-EO: Interpretable Code-based assistant for Earth Observation
  7. 2025Earth AI: Unlocking Geospatial Insights with Foundation Models and Cross-Modal Reasoning
  8. 2026Google Earth AI Geospatial Reasoning Agent: Capabilities and a Systematic Evaluation